Unicode Decode Error

[anonymous]

We have the following environment:

• RHEL 4
• trac 0.10.4
• svn 1.4.3

This ​link said that the following may be useful:

Python 2.3.4 (#1, Jan  9 2007, 16:40:18) 
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import trac.db.sqlite_backend
>>> trac.db.sqlite_backend._ver
(2, 8, 17)
>>> trac.db.sqlite_backend.have_pysqlite
1
>>> trac.db.sqlite_backend.sqlite.version
'1.0.1'
>>> 

When we run sqlite2pg we get the following error:

# ./sqlite2pg -e </path/to/trac> -p <postgres-connection-string>
Traceback (most recent call last):
  File "./sqlite2pg", line 331, in ?
    Main(opts)
  File "./sqlite2pg", line 250, in Main
    rc = tmigration.migrateTable(tname) or rc
  File "./sqlite2pg", line 74, in migrateTable
    rc = m()
  File "./sqlite2pg", line 105, in migrate_TICKET
    self.default_copy('ticket')
  File "./sqlite2pg", line 90, in default_copy
    for row in scur:
  File "/usr/lib/python2.3/site-packages/trac/db/util.py", line 40, in __iter__
    row = self.cursor.fetchone()
  File "/usr/lib/python2.3/site-packages/trac/db/sqlite_backend.py", line 73, in fetchone
    return row and self._convert_row(row) or None
  File "/usr/lib/python2.3/site-packages/trac/db/sqlite_backend.py", line 69, in _convert_row
    return tuple([(isinstance(v, str) and [v.decode('utf-8')] or [v])[0]
UnicodeDecodeError: 'utf8' codec can't decode byte 0x93 in position 59: unexpected code byte

Please advise as to what other information would be helpful in debugging this.

[ryan@…]

The patch in ​this ticket fixed our problem.

[John Hampton]

I'm not going to try to fix this for a few reasons.

1. You should really be using newer version of sqlite and pysqlite with 0.10.
2. There is a fix available as you mentioned
3. The fix isn't going to be integrated into trac proper
4. I agree with jonas, this situation shouldn't be encountered and if it is, then an evaluation of what went wrong needs to be done, rather than just blindly replacing data.

Thank you for the bug report and finding the fix, though.