Modify ↓

Opened 6 years ago

Closed 4 months ago

Last modified 4 months ago

#13487 closed defect (fixed)

KeyError: 'uid' - When browsing "Users" section in Account Manager

Reported by: totalcaos@… Owned by: c0redumb
Priority: high Component: LDAPAcctMngrPlugin
Severity: major Keywords: needinfo
Cc: totalcaos Trac Release: 1.2

Description (last modified by Ryan J Ollos)

When browsing to the users section in account manager i see this error: 

Trac detected an internal error:
KeyError: 'uid'

Tracelog:

File "build/bdist.linux-x86_64/egg/security/ldapstore.py", line 59, in get_users
Code fragment:
	
        try:
51	            con = self.init_connection()
52	            resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn','uid'])
53	        finally:
54	            if con != None:
55	                con.unbind()
56	       
57	        self.log.debug('List users: get %d users' % (len(resp)))
58	        for entry in resp:
59	            if entry[1]['uid'][0]:
60	                yield entry[1]['uid'][0]

The issue is that the user_matchfilter = sAMAccountName=%s not the default uid which i believe is hard referenced in your code (line 52)

Is there a way to make this more generic?

Attachments (2)

t13487-v2.diff​ (5.2 KB) - added by anonymous 6 years ago.
ldapstore-ptchd.py​ (4.4 KB) - added by Tony Albers 4 months ago.
Manually applied patch 13487-v2 to ldapstore.py

Download all attachments as: .zip

Change History (16)

comment:1 Changed 6 years ago by Ryan J Ollos

Description: modified (diff)

comment:2 Changed 6 years ago by Jun Omae

I think that it should add an option to specify LDAP field to match the username and authenticate rather than a part of LDAP filter.

Also, the username to use in LDAP filter should be escaped using ​ldap.filter.filter_format().

Untested patch:

  • ldapacctmngrplugin/trunk/ldapacctmngrplugin/security/ldapstore.py 

    y/ldapstore.py
index 2ee361972..6d119eac3 100644
    a b class LDAPStore (Component): 
    1111    bind_passwd = Option('ldap', 'bind_passwd', '', doc='For server not accepting anonymous bind, specify a bind_dn and password.')
    1212    user_searchbase = Option('ldap', 'user_searchbase', 'dc=company,dc=com', doc='Where to look for users. It is usually "dc=your_company_name,dc=com". Please consult the structure of your LDAP tree.')
    1313    user_searchfilter = Option('ldap', 'user_searchfilter', 'objectClass=inetOrgPerson', doc='Filter for listing valid users. The default ("objectClass=inetOrgPerson") should work for most of the cases.')
    14     user_matchfilter = Option('ldap', 'user_matchfilter', 'uid=%s', doc='The LDAP field for matching username when authenticating. The query is almost always "uid=%s".')
     14    user_field = Option('ldap', 'user_field', 'uid',
     15        doc="""The LDAP field to match the username and authenticate.""")
    1516
    1617    implements(IPasswordStore)
    1718
    … … class LDAPStore (Component): 
    1920        # Authenticate a user by checking password
    2021        con = None
    2122        base = self.user_searchbase
    22         filter = self.user_matchfilter % user
     23        user_field = self.user_field
     24        filter_ = ldap.filter.filter_format('%s=%s', [user_field, user])
    2325
    2426        # nested "try:" for python2.4
    2527        try:
    2628            try:
    2729                con = self.init_connection()
    28                 resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn'])
     30                resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter_, ['dn'])
    2931
    3032                # Added to prevent empty password authentication (some server allows that)
    3133                if not len(resp) :
    … … class LDAPStore (Component): 
    4446        # Get list of users from LDAP server
    4547        con = None
    4648        base = self.user_searchbase
     49        user_field = self.user_field
    4750        filter = self.user_searchfilter
    4851        resp = None
    4952
    5053        try:
    5154            con = self.init_connection()
    52             resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter, ['dn','uid'])
     55            resp = con.search_s(base, ldap.SCOPE_SUBTREE, filter,
     56                                ['dn', user_field])
    5357        finally:
    5458            if con != None:
    5559                con.unbind()
    5660
    5761        self.log.debug('List users: get %d users' % (len(resp)))
    5862        for entry in resp:
    59             if entry[1]['uid'][0]:
    60                 yield entry[1]['uid'][0]
     63            value = entry[1][user_field][0]
     64            if value:
     65                yield value
    6166
    6267    def init_connection(self):
    6368        # Initialize LDAP connection

comment:3 in reply to:  2 ; Changed 6 years ago by totalcaos@…

Thanks for the patch,

I modifed the ldapstore.py based on your patch, and have this config in trac.ini: 

[ldap]
bind_anonymous = no
bind_dn = CN=srv_ldap,CN=ServiceAccounts,CN=Users,DC=corp,DC=xxxx,DC=com
bind_passwd = XXXX
bind_server = ldap://xxxx:389
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName=%s
user_searchbase = CN=Staff,CN=Users,DC=corp,DC=xxx,DC=com
user_searchfilter = objectClass=person

I no longer see the KeyError: 'uid' when going to Admin -> Users, but don't see a list of my LDAP users. What am I doing that not correct?

How do I help you test this patch?

Thanks!

Last edited 6 years ago by Jun Omae (previous) (diff)

comment:4 Changed 6 years ago by totalcaos

Cc: totalcaos added

comment:5 in reply to:  3 Changed 6 years ago by Jun Omae

Replying to totalcaos@…:

I modifed the ldapstore.py based on your patch, and have this config in trac.ini: 

[ldap]
bind_anonymous = no
bind_dn = CN=srv_ldap,CN=ServiceAccounts,CN=Users,DC=corp,DC=xxxx,DC=com
bind_passwd = XXXX
bind_server = ldap://xxxx:389
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName=%s
user_searchbase = CN=Staff,CN=Users,DC=corp,DC=xxx,DC=com
user_searchfilter = objectClass=person

I no longer see the KeyError: 'uid' when going to Admin -> Users, but don't see a list of my LDAP users. What am I doing that not correct?

The user_field option should be sAMAccountName, not sAMAccountName=%s. 

user_field = sAMAccountName

Changed 6 years ago by anonymous

Attachment: t13487-v2.diff​ added

comment:6 Changed 6 years ago by Jun Omae

The patch is revised and tested with Active Directory 2012 R2: t13487-v2.diff​

Could you please try the patch?

P.S. I noticed the LDAPStore authenticates wrongly any username with empty password. The patch includes fix for this.

comment:7 Changed 6 years ago by Jun Omae

Here is configuration for the testing. 

[ldap]
bind_anonymous = no
bind_dn = administrator@DOMAIN.REALM
bind_passwd = passphrase
bind_server = ldap://DOMAIN.REALM/
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName
user_searchbase = dc=DOMAIN,dc=REALM
user_searchfilter = &(objectClass=user)(!(objectClass=computer))

comment:8 Changed 5 years ago by Jun Omae

Keywords: needinfo added

comment:9 Changed 4 months ago by Tony Albers

I was just hit by this. Running: Trac1.6 TracAccountManager 0.6.dev0 TracPermRedirect 3.0

And python 3.11

This is the error I get in the log: 

2023-12-14 13:08:00,841 Trac[ldapstore] DEBUG: List users: get 389 users
2023-12-14 13:08:00,844 Trac[main] ERROR: [10.23.245.35] Internal Server Error: <RequestWithSession 
"GET '/admin/accounts/users'">, referrer 'https://amgtrac01.jtl.local/admin'
Traceback (most recent call last):
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 609, in dispatch_requ
est
    dispatcher.dispatch(req)
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 301, in dispatch
    raise e
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 247, in dispatch
    resp = chosen_handler.process_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/admin/web_ui.py", line 104, in process_request
    resp = provider.render_admin_panel(req, cat_id, panel_id, path_info)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 218, in render_admin_panel
    return self._do_users(req)
           ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 690, in _do_users
    data.update(self._paginate(req, fetch_user_data(env, req,
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 47, in fetch_user_data
    for username in acctmgr.get_users():
                    ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/api.py", line 247, in get_users
    users.extend(store.get_users())
  File "/var/www/tracinst/lib/python3.11/site-packages/security/ldapstore.py", line 58, in get_users
    if entry[1]['uid'][0]:
       ~~~~~~~~^^^^^^^
KeyError: 'uid'

And I have this in trac.ini: 

[account-manager]
account_changes_notify_addresses = 
allow_delete_account = disabled
auth_init = disabled
cookie_refresh_pct = 10
db_htdigest_realm = 
environ_auth_overwrite = enabled
force_passwd_change = disabled
generated_password_length = 32
htdigest_file = /var/www/trac0.htpasswd
htdigest_realm = trac0
login_attempt_max_count = 6
login_opt_list = False
notify_actions = 
password_store = LDAPStore,HtDigestStore
persistent_sessions = disabled
refresh_passwd = disabled
register_check = 
reset_password = disabled
user_lock_max_time = 86400
user_lock_time = 300
user_lock_time_progression = 1
username_char_blacklist = :[]
verify_email = disabled

[components]
acct_mgr.admin.configurationadminpanel = enabled
acct_mgr.admin.useradminpanel = enabled
acct_mgr.api.accountmanager = enabled
acct_mgr.guard.accountguard = enabled
acct_mgr.htfile.htdigeststore = enabled
acct_mgr.model.* = enabled
acct_mgr.notification.accountchangelistener = enabled
acct_mgr.notification.accountchangenotificationadminpanel = enabled
acct_mgr.notification.accountnotificationformatter = enabled
acct_mgr.pwhash.htdigesthashmethod = enabled
acct_mgr.register.emailverificationmodule = disabled
acct_mgr.register.registrationmodule = disabled
acct_mgr.web_ui.loginmodule = enabled
acct_mgr.web_ui.resetpwstore = disabled
permredirect.filter.permredirectmodule = enabled
security.ldapstore.ldapstore = enabled
trac.web.auth.loginmodule = disabled

[ldap]
bind_anonymous = no
bind_dn = cn=BINDUSER,ou=ServiceAccounts,dc=example,dc=com
bind_passwd = PassWordS
bind_server = ldap://192.168.22.15:389/
user_matchfilter = sAMAccountName=%s
user_field = sAMAccountName
user_searchbase = ou=users,dc=example,dc=com
user_searchfilter = (&(objectClass=user)(!(objectClass=computer)))

Any help/advice is appreciated..

comment:10 in reply to:  9 Changed 4 months ago by Jun Omae

Replying to Tony Albers:

I was just hit by this. Running: Trac1.6 TracAccountManager 0.6.dev0 TracPermRedirect 3.0

And python 3.11

This is the error I get in the log: 

  File "/var/www/tracinst/lib/python3.11/site-packages/security/ldapstore.py", line 58, in get_users
    if entry[1]['uid'][0]:
       ~~~~~~~~^^^^^^^
KeyError: 'uid'

Could you please retry after applying t13487-v2.diff​ if the patch not applied?

Changed 4 months ago by Tony Albers

Attachment: ldapstore-ptchd.py​ added

Manually applied patch 13487-v2 to ldapstore.py

comment:11 Changed 4 months ago by Tony Albers

Hi Jun,

Thanks a lot for getting back to me.

Sure. However I had issues with 'patch' so I've done my best to apply the patch manually. (I've attached the resulting ldapstore.py).

This gives other errors, but it looks like the ldapstore plugin itself is pretty much working: 

2023-12-15 06:59:19,959 Trac[loader] DEBUG: Loading plugin "security.ldapstore" from "/var/www/traci
nst/lib/python3.11/site-packages"
...
2023-12-15 06:59:20,221 Trac[main] DEBUG: Dispatching <RequestWithSession "GET '/admin/accounts/users'">
2023-12-15 06:59:20,223 Trac[main] DEBUG: Chosen handler is <Component trac.admin.web_ui.AdminModule>
2023-12-15 06:59:20,224 Trac[session] DEBUG: Retrieving session for ID 'admin'
2023-12-15 06:59:20,226 Trac[api] DEBUG: action controllers for ticket workflow: ['ConfigurableTicketWorkflow']
2023-12-15 06:59:20,229 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing ACCTMGR_USER_ADMIN on None
2023-12-15 06:59:20,231 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing ACCTMGR_CONFIG_ADMIN on None
2023-12-15 06:59:20,232 Trac[main] DEBUG: Negotiated locale: None -> en_GB
2023-12-15 06:59:20,236 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/basics'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/logging'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing PERMISSION_GRANT on <Resource 'admin:general/perm'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TRAC_ADMIN on <Resource 'admin:general/plugin'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/components'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing MILESTONE_ADMIN on <Resource 'admin:ticket/milestones'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/versions'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/priority'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/resolution'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/severity'>
2023-12-15 06:59:20,237 Trac[perm] DEBUG: DefaultPermissionPolicy allows admin performing TICKET_ADMIN on <Resource 'admin:ticket/type'>
2023-12-15 06:59:20,264 Trac[main] ERROR: [10.23.245.35] Internal Server Error: <RequestWithSession "GET '/admin/accounts/users'">, referrer None
Traceback (most recent call last):
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 609, in dispatch_request
    dispatcher.dispatch(req)
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 301, in dispatch
    raise e
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/web/main.py", line 247, in dispatch
    resp = chosen_handler.process_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/admin/web_ui.py", line 104, in process_request
    resp = provider.render_admin_panel(req, cat_id, panel_id, path_info)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 218, in render_admin_panel
    return self._do_users(req)
           ^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 690, in _do_users
    data.update(self._paginate(req, fetch_user_data(env, req,
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/admin.py", line 53, in fetch_user_data
    if guard.user_locked(username):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/guard.py", line 219, in user_locked
    not user_known(self.env, user):
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/acct_mgr/model.py", line 318, in user_known
    for _ in env.db_query("""
             ^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/api.py", line 50, in execute
    return db.execute(query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/util.py", line 129, in execute
    cursor.execute(query, params if params is not None else [])
  File "/var/www/tracinst/lib/python3.11/site-packages/trac/db/util.py", line 73, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedFunction: operator does not exist: text = bytea
LINE 4:             WHERE authenticated=1 AND sid='\x7874686e'::byte...
                                                 ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
UndefinedFunction: operator does not exist: text = bytea
LINE 4:             WHERE authenticated=1 AND sid='\x7874686e'::byte...
                                                 ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.

/tony

comment:12 Changed 4 months ago by Jun Omae

Resolution: → fixed
Status: new → closed

In 18603:

Add option for user field and make compatible with Python 3 (closes #13487)

comment:13 Changed 4 months ago by Jun Omae

Please try the latest from repository.

comment:14 Changed 4 months ago by Tony Albers

Hi Jun,

I can confirm it's working as expected.

Thanks a lot for your help.

/tony

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain c0redumb.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment

E-mail address and name can be saved in the Preferences.

Attachments ↑ Description ↑
 
Note: See TracTickets for help on using tickets.