Opened 3 years ago

Closed 20 months ago

Ordinal out of range when including UTF-8 source

Reported by: Owned by: olaf.meeuwissen@… Ryan J Ollos normal IncludeMacro normal 1.0

Description

I'm trying to include a file from our SVN repository that contains UTF-8 Japanese. The svn:mime-type is set to text/x-trac-wiki. Here is what I (approximately) get:

Error: Macro Include(source:path/to/japanese.txt) failed

ascii<div class="code"><pre>{{{ #!comment 日本語のコメントです。 }}} これはうまくいかないね。
</pre></div>3756ordinal not in range(128)


The number before the ordinal not in range error message depends on the file's content.

Pasting the file's content into the wiki page works fine. Including wiki pages with Japanese text works fine too. I had expected the include macro to work fine as well.

comment:1 Changed 3 years ago by Ryan J Ollos

Status: new → assigned

I tried to reproduce, but I'm unable to. Are you running 3.0dev-r12030 of the plugin? Are you running Trac 1.0 or 1.0.1?

Would it be possible for you to post the content of japanese.txt that produces the error?

Last edited 3 years ago by Ryan J Ollos (previous) (diff)

comment:2 Changed 3 years ago by olaf.meeuwissen@…

Running Trac 1.0.1 with the 3.0.0dev version of the plugin. I downloaded the zip via the wiki page, so I don't have the exact version number handy. The zip file name included a 134546, so I guess I'm at or after r12030.

The only other plugin I have enabled at the moment is TracAccountManager 0.4.3. Babel is at 0.9.6 and our Trac environment has no other encoding issues (that I'm aware of). Subversion is 1.6.17 (r1128011). File I'm trying to include is attached.

Not sure if it matters but the server itself does not have any localization support installed (Debian wheezy, C system locale, no locales package, all upgrades applied as of writing).

Changed 3 years ago by olaf.meeuwissen@…

File I'm trying to include that exhibits the reported behaviour

comment:3 follow-up:  4 Changed 3 years ago by Ryan J Ollos

I can reproduce with Genshi 0.7.0, but not 0.6.1. Are you running Genshi 0.7.0?

I'm not sure how to fix the issue, but I'll continue investigating.

comment:4 in reply to:  3 Changed 3 years ago by olaf.meeuwissen@…

I can reproduce with Genshi 0.7.0, but not 0.6.1. Are you running Genshi 0.7.0?

Yes. FWIW, the instance runs in a Python virtual env and the "About Trac" page has:

Package Version
Trac 1.0.1
Babel 0.9.6
Genshi 0.7 (without speedups)
mod_wsgi 3.3 (WSGIProcessGroup project-name WSGIApplicationGroup localhost:8080|/project-name)
pysqlite 2.6.3
Python 2.6.8 (unknown, Jan 26 2013, 15:02:25) [GCC 4.7.2]
setuptools 0.6c11
SQLite 3.7.13
Subversion 1.6.17 (r1128011)
jQuery 1.7.2

I'm not sure how to fix the issue, but I'll continue investigating.

Thanks!

Last edited 3 years ago by Ryan J Ollos (previous) (diff)

comment:5 Changed 3 years ago by Ryan J Ollos

I would suggest downgrading for the time being, to Genshi 0.6.1, if you wish to use this plugin. I spent a few hours trying to fix the issue, but I am not sure how to fix it yet, and it could be a while before I can devote more time to it.

comment:6 Changed 3 years ago by olaf.meeuwissen@…

Thanks for your efforts so far and the feedback but I'm not sure that I can roll the affected Trac instance back to Genshi 0.6.1 any time soon, if at all.

comment:7 Changed 20 months ago by Ryan J Ollos

Resolution: → duplicate assigned → closed

It seems to work following the changes in #12422.

comment:8 Changed 20 months ago by anonymous

Thanks! The file is included without error. Now I just need to figure out why the file gets stuck in a <pre> block despite its MIME property but that's another issue.

Modify Ticket

Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted. Next status will be 'reopened'.