Modify

Opened 5 years ago

Closed 2 years ago

#10233 closed defect (fixed)

UnicodeDecode error when saving hours as csv (fix included)

Reported by: Dolf Andringa Owned by: Ryan J Ollos
Priority: high Component: TracHoursPlugin
Severity: critical Keywords: unicode csv export
Cc: Trac Release: 0.12

Description (last modified by Ryan J Ollos)

When downloading a csv file from the /hours url, sometimes I get a

UnicodeEncodeError: 'ascii' codec can't encode character ... in position 21: ordinal not in range(128)

This error occurs when a user entered unicode characters in tickets/hour logs. When downloading the csv file in queryhours2csv, this error was thrown.

The included diff fixes the error by properly encoding all strings as utf-8.

Attachments (1)

fix.diff (1.4 KB) - added by Dolf Andringa 5 years ago.

Download all attachments as: .zip

Change History (13)

Changed 5 years ago by Dolf Andringa

Attachment: fix.diff added

comment:1 Changed 5 years ago by Dolf Andringa

Severity: normalcritical

comment:2 Changed 5 years ago by Dolf Andringa

Priority: normalhigh

comment:3 Changed 5 years ago by Ryan J Ollos

Duplicate of #8520 I think, but thanks for the patch. I will be able to test and apply it later this week.

#8520 closed as a duplicate.

comment:4 Changed 4 years ago by Ryan J Ollos

Status: newassigned

I'm having trouble reproducing the issue. Could you provide a screen capture from just before the export, as well as the traceback you get when exporting?

comment:5 Changed 4 years ago by Ryan J Ollos

Here is the traceback that results:

How to Reproduce

While doing a GET operation on /hours, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'format': u'csv'}

User agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22

System Information

Trac 1.0.1
Babel 0.9.6
Genshi 0.6 (without speedups)
pysqlite 2.6.0
Python 2.7.3 (default, Aug 1 2012, 05:16:07)
[GCC 4.6.3]
setuptools 0.6
SQLite 3.7.9
jQuery 1.7.2

Enabled Plugins

ComponentDependencyPlugin 0.1
PrivateReports 0.4dev
TicketSidebarProvider 0.0-r11698
TracHoursPlugin 0.6.0dev

Python Traceback

Traceback (most recent call last):
  File "/home/user/Workspace/th11047/trac-1.0.1/trac/web/main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "/home/user/Workspace/th11047/trac-1.0.1/trac/web/main.py", line 214, in dispatch
    resp = chosen_handler.process_request(req)
  File "/home/user/Workspace/trachacks2.git/trachoursplugin/trunk/trachours/hours.py", line 240, in process_request
    return self.process_timeline(req)
  File "/home/user/Workspace/trachacks2.git/trachoursplugin/trunk/trachours/hours.py", line 522, in process_timeline
    return self.display_html(req, query)
  File "/home/user/Workspace/trachacks2.git/trachoursplugin/trunk/trachours/hours.py", line 826, in display_html
    self.queryhours2csv(req, data)
  File "/home/user/Workspace/trachacks2.git/trachoursplugin/trunk/trachours/hours.py", line 1032, in queryhours2csv
    for header in data['headers']])
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-9: ordinal not in range(128)

comment:6 Changed 4 years ago by Ryan J Ollos

#10848 closed as a duplicate.

comment:7 Changed 4 years ago by Ryan J Ollos

Resolution: fixed
Status: assignedclosed

(In [13210]) Fixes #10233: Encode data as utf-8 when exporting as CSV.

comment:8 in reply to:  description Changed 4 years ago by Ryan J Ollos

Replying to dolfandringa:

This error occurs when a user entered unicode characters in tickets/hour logs. When downloading the csv file in queryhours2csv, this error was thrown.

I think it can only happen if the ticket user, summary or similar field contains a unicode character. I can't see that the work comments ever get exported as CSV, regardless of how the query is setup. Please let me know if I'm overlooking something though, or if you can reproduce the issue still after 0.6.0dev-r13210.

comment:9 Changed 4 years ago by Ryan J Ollos

Description: modified (diff)
Keywords: unicode csv export added

comment:10 Changed 2 years ago by seqira@…

Resolution: fixed
Status: closedreopened

The problem still exists if there are headers in UTF (e.g. if trac is localized).

I'm not good at python (it's my first python code), but following changes fixed problem in current trunk.

  • trachours/hours.py

     
    968968        for groupname, results in data['groups']:
    969969            if groupname:
    970970                writer.writerow(unicode(groupname))
    971             writer.writerow([unicode(header['label'])
    972                              for header in data['headers']])
     971
     972
     973            #writer.writerow([unicode(header['label'])
     974            #                 for header in data['headers']])
     975            rowhead = []
     976            for header in data['headers']:
     977                rowhead.append(unicode(header['label']).encode('utf-8'))
     978            writer.writerow(rowhead)
    973979            for result in results:
    974980                row = []
    975981                for header in data['headers']:

comment:11 Changed 2 years ago by Ryan J Ollos

Status: reopenedaccepted

comment:12 Changed 2 years ago by Ryan J Ollos

Resolution: fixed
Status: acceptedclosed

In 14310:

0.6.0dev: Encode headers when exporting as CSV to fix traceback. Fixes #10233.

Modify Ticket

Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.