[patch] Tagged pages with names containing non-ASCII chars break listing for their tags

[claudius.link@…]

I love to use Trac and the Tag Plugin. As I use different languages for some pages I have pages which have non ascii characters (Umlauts) in some names. If these pages are tagged as well, they break the listing of pages having the same tags.

Reproduce

Create a page named PageWithUmlaut_Ü? with the content

[[ListTagged(ReproduceTag)]]

and tag it with ReproduceTag. Saving it you get

Error: Macro ListTagged(ReproduceTag) failed

asciiPageWithUmlaut_Ü1516ordinal not in range(128)

instead the list of tagged pages.

And as well in [tags?q='ReproduceTag']

Trac detected an internal error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xdc' in position 15: ordinal not in range(128)

This happens in ListTaggedMacro.expand_macro() in sorted() when applying str() to the unicode id. Simply removing the str() fixes the issue but messes up the sorting of the pages.

I don't understand why the problem occurs in the first place (and why I get two different error messages). I expected str() to work on UTF8. Setting default_charset = utf-8 in the trac.ini didn't help.

Nevertheless I attached the patch.

[claudius.link@…]

Patch from within tagsplugin/tags/0.6

[claudius.link@…]

I forgot the version Trac: 0.11.7 Python: 2.5.4 (r254:67916, Feb 11 2010, 00:50:55) [GCC 4.2.1 (Apple Inc. build 5646)] Genshi: 0.5.1

but I believe I see the same problem as well on Trac: 0.12dev-r8263 Python: 2.4.4 (#1, Oct 22 2008, 20:17:05) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] (but as that is my production wiki I didn't look so closely into it there)

[chenyuejie@…]

I have the same problem for wiki with name in Chinese. Trac 0.11.1

[lndr]

the patch works! ​http://trac-hacks.org/attachment/ticket/6908/listTaggedWithUmlaut.patch

Thanks a lot

[Steffen Hoffmann]

Replying to lndr:

the patch works! ​http://trac-hacks.org/attachment/ticket/6908/listTaggedWithUmlaut.patch

Thanks a lot

Nevertheless it's certainly not the preferred way to fix this issue.

I've seen, this issue has been fixed in trunk with changeset [3875] more than 2 years ago (see #2493), and missing backport to other branches has already be reported with #5575 as well.

So let's only keep one ticket open for tracking the remaining work. Anyway, thank you for taking your time to report and suggest a solution.