KeyError: 'uid' - When browsing "Users" section in Account Manager

[totalcaos@…]

When browsing to the users section in account manager i see this error:

Trac detected an internal error:
KeyError: 'uid'

Tracelog:

File "build/bdist.linux-x86_64/egg/security/ldapstore.py", line 59, in get_users
Code fragment:
	
        try:
51	            con = self.init_connection()
52	            resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn','uid'])
53	        finally:
54	            if con != None:
55	                con.unbind()
56	       
57	        self.log.debug('List users: get %d users' % (len(resp)))
58	        for entry in resp:
59	            if entry[1]['uid'][0]:
60	                yield entry[1]['uid'][0]

The issue is that the user_matchfilter = sAMAccountName=%s not the default uid which i believe is hard referenced in your code (line 52)

Is there a way to make this more generic?

[Jun Omae]

I think that it should add an option to specify LDAP field to match the username and authenticate rather than a part of LDAP filter.

Also, the username to use in LDAP filter should be escaped using ​ldap.filter.filter_format().

Untested patch:

ldapacctmngrplugin/trunk/ldapacctmngrplugin/security/ldapstore.py

@@ -11 +11 @@
     bind_passwd = Option('ldap', 'bind_passwd', '', doc='For server not accepting anonymous bind, specify a bind_dn and password.')
     user_searchbase = Option('ldap', 'user_searchbase', 'dc=company,dc=com', doc='Where to look for users. It is usually "dc=your_company_name,dc=com". Please consult the structure of your LDAP tree.')
     user_searchfilter = Option('ldap', 'user_searchfilter', 'objectClass=inetOrgPerson', doc='Filter for listing valid users. The default ("objectClass=inetOrgPerson") should work for most of the cases.')
-    user_matchfilter = Option('ldap', 'user_matchfilter', 'uid=%s', doc='The LDAP field for matching username when authenticating. The query is almost always "uid=%s".')
+    user_field = Option('ldap', 'user_field', 'uid',
+        doc="""The LDAP field to match the username and authenticate.""")
 
     implements(IPasswordStore)
 
@@ -19 +20 @@
         # Authenticate a user by checking password
         con = None
         base = self.user_searchbase
-        filter = self.user_matchfilter % user
+        user_field = self.user_field
+        filter_ = ldap.filter.filter_format('%s=%s', [user_field, user])
 
         # nested "try:" for python2.4
         try:
             try:
                 con = self.init_connection()
-                resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn'])
+                resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter_, ['dn'])
 
                 # Added to prevent empty password authentication (some server allows that)
                 if not len(resp) :
@@ -44 +46 @@
         # Get list of users from LDAP server
         con = None
         base = self.user_searchbase
+        user_field = self.user_field
         filter = self.user_searchfilter
         resp = None
 
         try:
             con = self.init_connection()
-            resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn','uid'])
+            resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter,
+                                ['dn', user_field])
         finally:
             if con != None:
                 con.unbind()
 
         self.log.debug('List users: get %d users' % (len(resp)))
         for entry in resp:
-            if entry[1]['uid'][0]:
-                yield entry[1]['uid'][0]
+            value = entry[1][user_field][0]
+            if value:
+                yield value
 
     def init_connection(self):
         # Initialize LDAP connection

[totalcaos@…]

Thanks for the patch,

I modifed the ldapstore.py based on your patch, and have this config in trac.ini:

[ldap]
bind_anonymous = no
bind_dn = CN=srv_ldap,CN=ServiceAccounts,CN=Users,DC=corp,DC=xxxx,DC=com
bind_passwd = XXXX
bind_server = ldap://xxxx:389
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName=%s
user_searchbase = CN=Staff,CN=Users,DC=corp,DC=xxx,DC=com
user_searchfilter = objectClass=person

I no longer see the KeyError: 'uid' when going to Admin -> Users, but don't see a list of my LDAP users. What am I doing that not correct?

How do I help you test this patch?

Thanks!

[Jun Omae]

Replying to totalcaos@…:

I modifed the ldapstore.py based on your patch, and have this config in trac.ini:

[ldap]
bind_anonymous = no
bind_dn = CN=srv_ldap,CN=ServiceAccounts,CN=Users,DC=corp,DC=xxxx,DC=com
bind_passwd = XXXX
bind_server = ldap://xxxx:389
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName=%s
user_searchbase = CN=Staff,CN=Users,DC=corp,DC=xxx,DC=com
user_searchfilter = objectClass=person

I no longer see the KeyError: 'uid' when going to Admin -> Users, but don't see a list of my LDAP users. What am I doing that not correct?

The user_field option should be sAMAccountName, not sAMAccountName=%s.

user_field = sAMAccountName

[Jun Omae]

The patch is revised and tested with Active Directory 2012 R2: t13487-v2.diff​

Could you please try the patch?

P.S. I noticed the LDAPStore authenticates wrongly any username with empty password. The patch includes fix for this.

[Jun Omae]

Here is configuration for the testing.

[ldap]
bind_anonymous = no
bind_dn = administrator@DOMAIN.REALM
bind_passwd = passphrase
bind_server = ldap://DOMAIN.REALM/
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName
user_searchbase = dc=DOMAIN,dc=REALM
user_searchfilter = &(objectClass=user)(!(objectClass=computer))

[Tony Albers]

I was just hit by this. Running: Trac1.6 TracAccountManager 0.6.dev0 TracPermRedirect 3.0

And python 3.11

This is the error I get in the log:

2023-12-14 13:08:00,841 Trac[ldapstore] DEBUG: List users: get 389 users
2023-12-14 13:08:00,844 Trac[main] ERROR: [10.23.245.35] Internal Server Error: <RequestWithSession 
"GET '/admin/accounts/users'">, referrer 'https://amgtrac01.jtl.local/admin'
Traceback (most recent call last):
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 609, in dispatch_requ
est
    dispatcher.dispatch(req)
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 301, in dispatch
    raise e
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 247, in dispatch
    resp = chosen_handler.process_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/admin/web_ui.py", line 104, in process_request
    resp = provider.render_admin_panel(req, cat_id, panel_id, path_info)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 218, in render_admin_panel
    return self._do_users(req)
           ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 690, in _do_users
    data.update(self._paginate(req, fetch_user_data(env, req,
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 47, in fetch_user_data
    for username in acctmgr.get_users():
                    ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/api.py", line 247, in get_users
    users.extend(store.get_users())
  File "/var/www/tracinst/lib/python3.11/site-packages/security/ldapstore.py", line 58, in get_users
    if entry[1]['uid'][0]:
       ~~~~~~~~^^^^^^^
KeyError: 'uid'

And I have this in trac.ini:

[account-manager]
account_changes_notify_addresses = 
allow_delete_account = disabled
auth_init = disabled
cookie_refresh_pct = 10
db_htdigest_realm = 
environ_auth_overwrite = enabled
force_passwd_change = disabled
generated_password_length = 32
htdigest_file = /var/www/trac0.htpasswd
htdigest_realm = trac0
login_attempt_max_count = 6
login_opt_list = False
notify_actions = 
password_store = LDAPStore,HtDigestStore
persistent_sessions = disabled
refresh_passwd = disabled
register_check = 
reset_password = disabled
user_lock_max_time = 86400
user_lock_time = 300
user_lock_time_progression = 1
username_char_blacklist = :[]
verify_email = disabled

[components]
acct_mgr.admin.configurationadminpanel = enabled
acct_mgr.admin.useradminpanel = enabled
acct_mgr.api.accountmanager = enabled
acct_mgr.guard.accountguard = enabled
acct_mgr.htfile.htdigeststore = enabled
acct_mgr.model.* = enabled
acct_mgr.notification.accountchangelistener = enabled
acct_mgr.notification.accountchangenotificationadminpanel = enabled
acct_mgr.notification.accountnotificationformatter = enabled
acct_mgr.pwhash.htdigesthashmethod = enabled
acct_mgr.register.emailverificationmodule = disabled
acct_mgr.register.registrationmodule = disabled
acct_mgr.web_ui.loginmodule = enabled
acct_mgr.web_ui.resetpwstore = disabled
permredirect.filter.permredirectmodule = enabled
security.ldapstore.ldapstore = enabled
trac.web.auth.loginmodule = disabled

[ldap]
bind_anonymous = no
bind_dn = cn=BINDUSER,ou=ServiceAccounts,dc=example,dc=com
bind_passwd = PassWordS
bind_server = ldap://192.168.22.15:389/
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName
user_searchbase = ou=users,dc=example,dc=com
user_searchfilter = (&(objectClass=user)(!(objectClass=computer)))

Any help/advice is appreciated..

[Jun Omae]

Replying to Tony Albers:

I was just hit by this. Running: Trac1.6 TracAccountManager 0.6.dev0 TracPermRedirect 3.0

And python 3.11

This is the error I get in the log:

  File "/var/www/tracinst/lib/python3.11/site-packages/security/ldapstore.py", line 58, in get_users
    if entry[1]['uid'][0]:
       ~~~~~~~~^^^^^^^
KeyError: 'uid'

Could you please retry after applying t13487-v2.diff​ if the patch not applied?

[Tony Albers]

Manually applied patch 13487-v2 to ldapstore.py

[Tony Albers]

Hi Jun,

Thanks a lot for getting back to me.

Sure. However I had issues with 'patch' so I've done my best to apply the patch manually. (I've attached the resulting ldapstore.py).

This gives other errors, but it looks like the ldapstore plugin itself is pretty much working:

2023-12-15 06:59:19,959 Trac[loader] DEBUG: Loading plugin "security.ldapstore" from "/var/www/traci
nst/lib/python3.11/site-packages"
...
2023-12-15 06:59:20,221 Trac[main] DEBUG: Dispatching <RequestWithSession "GET '/admin/accounts/users'">
2023-12-15 06:59:20,223 Trac[main] DEBUG: Chosen handler is <Component trac.admin.web_ui.AdminModule>
2023-12-15 06:59:20,224 Trac[session] DEBUG: Retrieving session for ID 'admin'
2023-12-15 06:59:20,226 Trac[api] DEBUG: action controllers for ticket workflow: ['ConfigurableTicketWorkflow']
2023-12-15 06:59:20,229 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing ACCTMGR_USER_ADMIN on None
2023-12-15 06:59:20,231 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing ACCTMGR_CONFIG_ADMIN on None
2023-12-15 06:59:20,232 Trac[main] DEBUG: Negotiated locale: None -> en_GB
2023-12-15 06:59:20,236 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/basics'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/logging'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing PERMISSION_GRANT on <Resource 'admin:general/perm'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/plugin'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/components'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing MILESTONE_ADMIN on <Resource 'admin:ticket/milestones'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/versions'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/priority'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/resolution'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/severity'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/type'>
2023-12-15 06:59:20,264 Trac[main] ERROR: [10.23.245.35] Internal Server Error: <RequestWithSession "GET '/admin/accounts/users'">, referrer None
Traceback (most recent call last):
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 609, in dispatch_request
    dispatcher.dispatch(req)
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 301, in dispatch
    raise e
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 247, in dispatch
    resp = chosen_handler.process_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/admin/web_ui.py", line 104, in process_request
    resp = provider.render_admin_panel(req, cat_id, panel_id, path_info)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 218, in render_admin_panel
    return self._do_users(req)
           ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 690, in _do_users
    data.update(self._paginate(req, fetch_user_data(env, req,
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 53, in fetch_user_data
    if guard.user_locked(username):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/guard.py", line 219, in user_locked
    not user_known(self.env, user):
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/model.py", line 318, in user_known
    for _ in env.db_query("""
             ^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/api.py", line 50, in execute
    return db.execute(query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/util.py", line 129, in execute
    cursor.execute(query, params if params is not None else [])
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/util.py", line 73, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedFunction: operator does not exist: text = bytea
LINE 4:             WHERE authenticated=1 AND sid='\x7874686e'::byte...
                                                 ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
UndefinedFunction: operator does not exist: text = bytea
LINE 4:             WHERE authenticated=1 AND sid='\x7874686e'::byte...
                                                 ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.

/tony

[Jun Omae]

In 18603:

Add option for user field and make compatible with Python 3 (closes #13487)

[Jun Omae]

Please try the latest from repository.

[Tony Albers]

Hi Jun,

I can confirm it's working as expected.

Thanks a lot for your help.

/tony