Modify

Opened 2 years ago

Closed 17 months ago

Last modified 17 months ago

#10139 closed defect (fixed)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 9-10: ordinal not in range(128)

Reported by: manuel.neto Owned by: sandinak
Priority: normal Component: DirectoryAuthPlugin
Severity: major Keywords: unicode encoding
Cc: rjollos Trac Release: 1.0

Description

How to Reproduce

While doing a GET operation on /admin/accounts/users, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'cat_id': u'accounts', 'panel_id': u'users', 'path_info': None}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1

System Information

Trac 0.12.3
Trac 0.12.3
Agilo 0.9.8
Babel 0.9.6
CustomFieldAdmin 0.2.6
Docutils 0.8.1
Genshi 0.6
mod_wsgi 3.3 (WSGIProcessGroup WSGIApplicationGroup trac2.atualtec.com|)
psycopg2 2.2.1
Pygments 1.4
Python 2.6.6 (r266:84292, Dec 26 2010, 22:48:11)
[GCC 4.4.5]
Python 2.6.6 (r266:84292, Dec 26 2010, 22:48:11)
[GCC 4.4.5]
pytz 2011n
pytz 2011n
RPC 1.1.2
setuptools 0.6c11
setuptools 0.6c11
Subversion 1.6.12 (r955767)
jQuery 1.4.4

Enabled Plugins

agilo 0.9.8
AutocompleteUsers 0.4.2
AutoUpgrade 0.1
DiskUsagePlugin 0.2
IniAdmin 0.2
LdapPlugin 0.6.0dev
siteupload 0.11
TracAccountManager 0.3.2
TracActiveDirectoryAuth 0.2.2
TracAnnouncer 0.11.1
TracCustomFieldAdmin 0.2.6
TracDragDrop 0.12.0.5
TracRevtreePlugin 0.6.4dev
TracWikiPrintPlugin 1.8.4
TracXMLRPC 1.1.2

Python Traceback

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.3-py2.6.egg/trac/web/main.py", line 522, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.3-py2.6.egg/trac/web/main.py", line 243, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.3-py2.6.egg/trac/admin/web_ui.py", line 117, in process_request
    path_info)
  File "build/bdist.linux-x86_64/egg/acct_mgr/admin.py", line 137, in render_admin_panel
    return self._do_users(req)
  File "build/bdist.linux-x86_64/egg/acct_mgr/admin.py", line 313, in _do_users
    for username in acctmgr.get_users():
  File "build/bdist.linux-x86_64/egg/acct_mgr/api.py", line 184, in get_users
    users.extend(store.get_users())
  File "build/bdist.linux-x86_64/egg/tracext/adauth/auth.py", line 53, in get_users
    'proxyAddresses', 'displayName'])
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 502, in search_s
    return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 495, in search_ext_s
    msgid = self.search_ext(base,scope,filterstr,attrlist,attrsonly,serverctrls,clientctrls,timeout,sizelimit)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 491, in search_ext
    timeout,sizelimit,
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 96, in _ldap_call
    result = func(*args,**kwargs)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 9-10: ordinal not in range(128)

Attachments (0)

Change History (11)

comment:1 Changed 2 years ago by hasienda

  • Keywords unicode encoding added

At least one point down this traceback there is wrong handling of non-ASCII/Unicode characters. But it's not obvious where yet. We need more information and look at all code involved to find and correct it.

comment:2 Changed 2 years ago by hasienda

Out fo curiosity: Is your LDAP backend guaranteed to properly handle Unicode characters?

comment:3 Changed 2 years ago by rjollos

  • Cc rjollos added

comment:4 Changed 20 months ago by hasienda

  • Keywords needinfo added

I can't do more without feedback. Please follow-up on my previous questions.

comment:5 Changed 20 months ago by hasienda

  • Component changed from AccountManagerPlugin to DirectoryAuthPlugin
  • Keywords needinfo removed
  • Owner changed from hasienda to sandinak

Just found out a bit more about the source of this issue, that makes me believe, that sandinak might be in a better position to fix it than me.

Rationale: The last caller is not AccountManagerPlugin, it's ActiveDirectoryAuthPlugin (today known as DirectoryAuthPlugin), a password store for AccountManagerPlugin that provides authentication and groups from Lightweight Directory Access Protocol (LDAP) enabled service.

Maybe this has already been fixed meanwhile, and upgrading your ActiveDirectoryAuthPlugin to a recent version of it's successor would help?

comment:6 Changed 18 months ago by sima.baymani@…

  • Severity changed from normal to major

I have Trac 1.0 installed on Ubuntu, using DirectoryAuthPlugin r12025 (HEAD on trunk at this date). I can verify that this problem still exists for users with non-ASCII characters in their full names.

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/api.py", line 502, in send_error
    data, 'text/html')
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/chrome.py", line 955, in render_template
    message = req.session.pop('chrome.%s.%d' % (type_, i))
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/api.py", line 304, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/main.py", line 268, in _get_session
    return Session(self.env, req)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/session.py", line 200, in __init__
    if req.authname == 'anonymous':
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/api.py", line 304, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0-py2.6.egg/trac/web/main.py", line 135, in authenticate
    authname = authenticator.authenticate(req)
  File "/usr/local/lib/python2.6/dist-packages/TracAccountManager-0.4-py2.6.egg/acct_mgr/util.py", line 45, in wrap
    return func(self, *args, **kwds)
  File "/usr/local/lib/python2.6/dist-packages/TracAccountManager-0.4-py2.6.egg/acct_mgr/web_ui.py", line 300, in authenticate
    user = self._remote_user(req)
  File "/usr/local/lib/python2.6/dist-packages/TracAccountManager-0.4-py2.6.egg/acct_mgr/web_ui.py", line 649, in _remote_user
    if acctmgr.check_password(user, password) == True:
  File "/usr/local/lib/python2.6/dist-packages/TracAccountManager-0.4-py2.6.egg/acct_mgr/api.py", line 259, in check_password
    valid = store.check_password(user, password)
  File "/usr/local/lib/python2.6/dist-packages/TracDirectoryAuth-0.5-py2.6.egg/tracext/adauth/auth.py", line 150, in check_password
    success = self._bind_dir(dn, password) or False
  File "/usr/local/lib/python2.6/dist-packages/TracDirectoryAuth-0.5-py2.6.egg/tracext/adauth/auth.py", line 256, in _bind_dir
    self.log.debug('_bind_dir: attempting specific bind to %s as %s' % (self.dir_uri, user_dn))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)

I have read a suggestion on the general problem,adding a 'u' to the format string - but this does not seem to help. (Search for "Django: How to fix the annoying UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)" as the spam filter prevented me from adding the link)

Using the same patch as in #10443 (str.encode('utf-8')) does not help either.

I'm at loss here and some users can't log in - any help or suggestion would be highly appreciated.

comment:7 Changed 17 months ago by sima.baymani@…

  • Trac Release changed from 0.11 to 1.0

Changing Trac release from 0.11 to 1.0, since it is there as well. I haven't had time to look much on it myself but it seems that some rework on the code may be needed on how strings are handled.

I'll pop in again if I find a solution.

comment:8 Changed 17 months ago by anonymous

  • Status changed from new to assigned

comment:9 Changed 17 months ago by anonymous

  • Resolution set to fixed
  • Status changed from assigned to closed

Researched and found http://blog.codekills.net/2008/05/01/encoding-and-decoding-text-in-python-%28or---i-didn%27t-ask-you-to-use-the-%27ascii%27-codec!-%29/

Which talks about this case. I don't have a Unicode DN to test with ;-) I have patched it for the next release .. I'd ask anonymoose to test .. but 'es not around.

Also checked any further printing of user_dn in the debug output .. none here.

comment:10 follow-up: Changed 17 months ago by hasienda

You might want to use a slightly more trac-ish markup for commit messages to help TicketCommitUpdater pick and add ticket comments like so:

(In [12719]) Fix UnicodeEncodeError?: 'ascii' codec can't encode characters in position 9-10: ordinal not in range(128), closes #10139.

Just use 'closes #<tkt_no>' (comment and resolve) or 'refs #<tkt_no>' (comment-only) to trigger actions. Added bonus are back-links to the changeset from as many tickets (at trac-hacks.org) as you like. Needless to say: Handle with care and double-check numbers because at least links from repository to tickets are immutable of course.

comment:11 in reply to: ↑ 10 Changed 17 months ago by rjollos

Replying to hasienda:

You might want to use a slightly more trac-ish markup for commit messages to help TicketCommitUpdater pick and add ticket comments like so: [...]

Yes, I would definitely recommend what hasienda suggested! It's one of the best features we have on this now ancient site :)

Add Comment

Modify Ticket

Action
as closed .
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from sandinak. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.