Modify

Opened 6 years ago

Closed 4 years ago

#4458 closed defect (fixed)

UnicodeDecodeError

Reported by: weltweitehrlich Owned by: farialima
Priority: normal Component: TicketImportPlugin
Severity: normal Keywords:
Cc: Trac Release: 0.11

Description

We have german umlauts. Importing a exported unmodified CSV generates:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 11: ordinal not in range(128)}}}

Attachments (1)

query.csv (122 bytes) - added by weltweitehrlich 6 years ago.
Example file for this error

Download all attachments as: .zip

Change History (13)

comment:1 follow-ups: Changed 6 years ago by farialima

CSV files and non-ascii codecs are tricky. I thought that it was working, but it seems that it's not perfect.

Could you attach a CSV file that exposes the problem ? Also, are you on Python 2.4, 2.5 or 2.6 ?

I think that an Excel spreadsheet would work - that's probably a workaround for you ?

Changed 6 years ago by weltweitehrlich

Example file for this error

comment:2 in reply to: ↑ 1 Changed 6 years ago by weltweitehrlich

Replying to farialima:

Could you attach a CSV file that exposes the problem ?

Done.

Also, are you on Python 2.4, 2.5 or 2.6 ?

I'm using python 2.5

I think that an Excel spreadsheet would work - that's probably a workaround for you ?

I have no xlrd installation at the moment. I will test it later.

comment:3 follow-up: Changed 6 years ago by giangnh@…

I met exactly same problem in Japanese(even use xlrd).

comment:4 in reply to: ↑ 1 ; follow-up: Changed 6 years ago by weltweitehrlich

Replying to farialima:

I think that an Excel spreadsheet would work - that's probably a workaround for you ?

The Excel spreadsheet seems to work. You are right. The ticket export is CSV so it's not the best process to import using Excel. Can you fix it?

comment:5 in reply to: ↑ 4 ; follow-up: Changed 6 years ago by farialima

Replying to weltweitehrlich:

The ticket export is CSV so it's not the best process to import using Excel. Can you fix it?

I've committed a fix and updated the ".egg" files on the Wiki page. Please try it and tell me if it works for you.

comment:6 in reply to: ↑ 3 Changed 6 years ago by farialima

Replying to giangnh@aureole-net.com.vn:

I met exactly same problem in Japanese(even use xlrd).

Could you attach a .csv file, and an .xls file, to this ticket, so that I can look at the problem ?

I don't think that the fix I just did will work for you: my fix supposes that the .csv file uses a European encoding. However, if you give me some sample files, I will try to fix it for Japanese also.

comment:7 in reply to: ↑ 5 Changed 6 years ago by weltweitehrlich

Replying to farialima:

Replying to weltweitehrlich:

The ticket export is CSV so it's not the best process to import using Excel. Can you fix it?

I've committed a fix and updated the ".egg" files on the Wiki page. Please try it and tell me if it works for you.

The error is gone. That's the good point. The bad one is that the encoding seems to be wrong. Maybe my fault. I must check this.

comment:8 Changed 5 years ago by farialima

Any news on this one ? can I close it as fixed ?

comment:9 follow-up: Changed 4 years ago by anonymous

tried import of a Japanese file. csv format doesn't work, but Excel 2003 works fine with xlrd.

comment:10 Changed 4 years ago by ruidpires@…

Problem still happening if the file has the character u\xea which translates to 'ê'

Other than that, it seems to be working.

comment:11 in reply to: ↑ 9 Changed 4 years ago by farialima

  • Status changed from new to assigned

Based on anonymous:

tried import of a Japanese file. csv format doesn't work, but Excel 2003 works fine with xlrd.

In #4458 I've added a warning:

"(Excel is preferred if your data contain non-ASCII characters). "

because there will be *always* issues with charsets in CSV - it does not support full Unicode.

I think we can mark this as fixed... anyone disagreeing can reopen it :)

comment:12 Changed 4 years ago by farialima

  • Resolution set to fixed
  • Status changed from assigned to closed

Add Comment

Modify Ticket

Action
as closed The owner will remain farialima.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.