Modify

Opened 19 months ago

Closed 19 months ago

Last modified 19 months ago

#11133 closed defect (cantfix)

TocMacro stopped working with trac 1.0.1

Reported by: damato Owned by: rjollos
Priority: normal Component: TocMacro
Severity: blocker Keywords: genshi 0.7
Cc: hasienda Trac Release: 1.0

Description

I recently recognized that the TocMacro somehow seem to have stopped working with my installed copy of trac 1.0.1. Here the following error is returned instead when a page is layouted which uses the TOC() macro:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 85: ordinal not in range(128)

The trac.log shows the following entries if that happens:

2013-06-02 13:16:16,974 Trac[formatter] ERROR: Macro TOC(titleindex, heading=FAQ Contents, notitle, depth=2, FAQ/General Questions, FAQ/Installing YAM, FAQ/Using YAM, FAQ/ARexx Scripting, FAQ/Debugging) failed:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 765, in _macro_formatter
    return macro.ensure_inline(macro.process(args))
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 356, in process
    text = self.processor(text)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 343, in _macro_processor
    text)
  File "build/bdist.linux-x86_64/egg/tractoc/macro.py", line 193, in expand_macro
    params['min_depth'] < 2)
  File "build/bdist.linux-x86_64/egg/tractoc/macro.py", line 215, in _render_title_index
    fmt.format(page_text, NullOut())
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1423, in format
    Formatter.format(self, text)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1283, in format
    result = re.sub(self.wikiparser.rules, self.replace, line)
  File "/usr/lib64/python2.7/re.py", line 151, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1201, in replace
    replacement = self.handle_match(fullmatch)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1197, in handle_match
    return internal_handler(match, fullmatch)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 579, in _shrefbr_formatter
    self._make_link(ns, target, match, match, fullmatch)
  File "/usr/lib64/python2.7/site-packages/genshi/builder.py", line 107, in __str__
    return str(self.generate())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 85: ordinal not in range(128)

If you want to have a look at the corresponding page which uses this macro have a look at:

https://yam.ch/wiki/FAQ

There you should be able to see the error in question.

Attachments (0)

Change History (9)

comment:1 Changed 19 months ago by rjollos

  • Owner changed from coderanger to rjollos
  • Status changed from new to assigned

You are running Genshi 0.7? We've seen this problem in #10989 and #11145.

comment:2 Changed 19 months ago by rjollos

  • Keywords genshi 0.7 added

comment:3 Changed 19 months ago by damato

yes, I am using genshi 0.7

comment:4 Changed 19 months ago by rjollos

The problem may not be with the TocMacro. Could you check if there is a traceback in the logs that results from accessing https://yam.ch/wiki/FAQ/Using%20YAM?

comment:5 Changed 19 months ago by rjollos

Note also,

  • On the AllInOne page, the wiki in the included page FAQ/Using YAM is not formatted to HTML.
  • When trying to export to PDF a similar error occurs, again in Formatter.format:

How to Reproduce

While doing a GET operation on /wiki/FAQ/Using YAM, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'format': u'pdf', 'page': u'FAQ/Using YAM'}

User agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22

System Information

System information not available

Enabled Plugins

Plugin information not available

Python Traceback

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/trac/web/main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib64/python2.7/site-packages/trac/web/main.py", line 214, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/web_ui.py", line 177, in process_request
    format, versioned_page.name)
  File "/usr/lib64/python2.7/site-packages/trac/mimeview/api.py", line 1015, in send_converted
    content, selector)
  File "/usr/lib64/python2.7/site-packages/trac/mimeview/api.py", line 691, in convert_content
    output = converter.convert_content(req, mimetype, content, ck)
  File "build/bdist.linux-x86_64/egg/pagetopdf/pagetopdf.py", line 22, in convert_content
    page = wiki_to_html(source, self.env, req).encode(codepage, 'replace')
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1572, in wiki_to_html
    Formatter(env, context).format(wikitext, out, escape_newlines)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1283, in format
    result = re.sub(self.wikiparser.rules, self.replace, line)
  File "/usr/lib64/python2.7/re.py", line 151, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1201, in replace
    replacement = self.handle_match(fullmatch)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 1197, in handle_match
    return internal_handler(match, fullmatch)
  File "/usr/lib64/python2.7/site-packages/trac/wiki/formatter.py", line 579, in _shrefbr_formatter
    self._make_link(ns, target, match, match, fullmatch)
  File "/usr/lib64/python2.7/site-packages/genshi/builder.py", line 107, in __str__
    return str(self.generate())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 85: ordinal not in range(128)

When moving the content into a development instance, I'm still unable to reproduce.

Last edited 19 months ago by rjollos (previous) (diff)

comment:6 follow-up: Changed 19 months ago by rjollos

  • Resolution set to cantfix
  • Status changed from assigned to closed

I couldn't reproduce because I was testing with Trac 1.0-stable. I was able to reproduce when switching to Trac 1.0.1.

The problem has been fixed in [t 11683]. It's too bad there isn't a ticket associated with that changeset.

As far as I can see, you have three options:

  1. Downgrade to Genshi 6.1
  2. Upgrade to Trac 1.0-stable
  3. Remove http from [wiki] safe_schema. The result will be that URLs aren't rendered as hyperlinks.
Version 1, edited 19 months ago by rjollos (previous) (next) (diff)

comment:7 follow-up: Changed 19 months ago by rjollos

  • Cc hasienda added; anonymous removed

Note in comment:6, a new resolution: cantfix. Trac has that resolution, and bugs filed against t.e.o that are plugin issues are frequently closed as cantfix. It seems appropriate and provides a nice sense of symmetry to do the same here when the problem is with Trac.

hasienda: does this sound okay to you?

comment:8 in reply to: ↑ 7 Changed 19 months ago by hasienda

Replying to rjollos:

Note in comment:6, a new resolution: cantfix. Trac has that resolution, and bugs filed against t.e.o that are plugin issues are frequently closed as cantfix. It seems appropriate and provides a nice sense of symmetry to do the same here when the problem is with Trac.

hasienda: does this sound okay to you?

I agree, even if occasions may be rare, where nothing could be done for a plugin to work around Trac core issues.

Few weeks ago I encountered some code, where I needed to replace '' with u'' myself, what made me think a bit more. Neither looked into the Genshi change nore heard much about about an upcoming Python2to3 effort by now, but this is definitely related, because one of the major changes is nothing-else-than-bytecode-or-unicode directive in Python3.

comment:9 in reply to: ↑ 6 Changed 19 months ago by damato

Replying to rjollos:

The problem has been fixed in [t 11683]. It's too bad there isn't a ticket associated with that changeset.

As far as I can see, you have three options:

  1. Downgrade to Genshi 6.1
  2. Upgrade to Trac 1.0-stable > t:r11683.
  3. Remove http from [wiki] safe_schema. The result will be that URLs aren't rendered as hyperlinks.

Please note, that I took the 4. possibility in simply applying the fix to formatter.py in my trac installation which is pointed out in t:r11683 :)

Now everything seems to work fine and the problem doesn't occurr anymore. I guess, its time to see 1.0.2 being released soon...

Add Comment

Modify Ticket

Action
as closed The owner will remain rjollos.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.