UnicodeDecodeError when creating PDF export

[anonymous]

Plugin activated by webadmin. I add few pages and click on create button :

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 356, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 224, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.4/site-packages/TracWebAdmin-0.1.2dev_r3772-py2.4.egg/webadmin/web_ui.py", line 109, in process_request
    path_info)
  File "build/bdist.linux-i686/egg/combinewiki/web_ui.py", line 60, in process_admin_request
  File "build/bdist.linux-i686/egg/combinewiki/web_ui.py", line 110, in _page_to_file
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 34: ordinal not in range(128)

[Noah Kantrowitz]

Did you set your charset to something (probably you mean to use UTF8)?

[jl]

I got the same error on a windows system.

What did you mean with "set your charset to something"?

[Noah Kantrowitz]

In trac.ini, what is default_charset set to?

[jl]

I tried the following things: By default is was set to iso-8859-15. I tried tio change it to UTF-8 same result.

But this is only happening with some wiki-sides:

• All my own sides are not working (same error as above)
• Some wiki pages are working: e.g. TracIni
• Some not working: e.g. TracInstall (see error message below)

Traceback (most recent call last):
  File "D:\PROGRA~1\Python23\Lib\site-packages\trac\web\main.py", line 356, in dispatch_request
    dispatcher.dispatch(req)
  File "D:\PROGRA~1\Python23\Lib\site-packages\trac\web\main.py", line 224, in dispatch
    resp = chosen_handler.process_request(req)
  File "build\bdist.win32\egg\webadmin\web_ui.py", line 109, in process_request
  File "build\bdist.win32\egg\combinewiki\web_ui.py", line 60, in process_admin_request
  File "build\bdist.win32\egg\combinewiki\web_ui.py", line 107, in _page_to_file
  File "D:\PROGRA~1\Python23\Lib\encodings\iso8859_15.py", line 18, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u201c' in position 10962: character maps to <undefined>

Also the links to wiki pages are not correct in the PDF, there is missing the /wiki/ in the URL.

[sdouche]

Hi, same error here with default sites (iso 8859-15, activated by WebAdmin or manually) with all wiki pages I have.

[frederic.duarte@…]

Greetings,

I've got the same error. Do you think that it could be realted to Trac ticket ​#3723 ?

[Noah Kantrowitz]

More likely it is due to your version of HTMLDOC. I still need to merge over Alec's latest changes to PageToPdf, which should fix this I think.

[Noah Kantrowitz]

(In [1490]) * Apply [1421] (refs #815), [1441], and [1462].

• Sort pages in the left pane (closes #876).

[Noah Kantrowitz]

Okay, which version of HTMLDOC do you have?

[sdouche]

Dapper's version : 1.8.24 (revision 2).

[Noah Kantrowitz]

Hmm, thats the same version as me. I have default_charset = utf-8, so maybe that will help?

[frederic.duarte]

Thank you for having closed #876 so fast, it works just fine.

But I still have the UnicodeDecodeError. My config is HTMLDoc 1.8.27 and I tryed default_charset as iso-8859-15 and utf-8 and the result is still the same.

[sdouche]

Hmm, thats the same version as me. I have default_charset = utf-8, so maybe that will help?

Same error after changed settings.

[Noah Kantrowitz]

Please see this comment.

[sdouche]

Oups ! Sorry i have not the same error, I read too fast :

File "build/bdist.linux-i686/egg/combinewiki/web_ui.py", line 60, in process_admin_request File "build/bdist.linux-i686/egg/combinewiki/web_ui.py", line 110, in _page_to_file

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 34: ordinal not in range(128)

It's not exactly same issue (Python 2.4, Trac 0.10, Ubuntu Dapper). Do I need create a new ticket ? (error occured after comment's change).

[Noah Kantrowitz]

You cannot use characters that would require a multi-byte charset.

[andy@…]

I got this problem again:

UnicodeEncodeError: 'charmap' codec can't encode character u'\u201c' in position 13810: character maps to <undefined>

and I found a ticket for the PageToPDFPlugin, that seems to solve the problem: #1076

[sdouche]

It's work for me with default_charset=utf-8 and iso in [pdftofile]. But isn't perfect. The Plugins translate with (in formats.py) :

page = wiki_to_html(text, self.env, req).encode(codepage)

It's not enough for a smooth translation : "événements" become "événements"

Maybe add a dictionnary like function to translate specific characters ?

[Noah Kantrowitz]

As is stated many times, HTMLDOC does not support any multibyte character sets. You simply cannot use them with the PDF output format. The HTML output format does work with Unicode, so you can use that and then a local PDF virtual printer if you want.