Handle unicode arguments for embedded custom TracQuery too

[bog@…]

I had tried 0.11 (as single file) and 0.12 (as egg install in .../plugins) and 0.12 (as easy_install). In any case i have:

Genshi UnicodeDecodeError error while rendering template (unknown template location)

CentOS 5.4 + ​http://packages.sw.be/trac/
Apache 2.2.3
trac 0.12
python-setuptools-0.6c5-2.el5
python-genshi-0.6-1.el5.rf
python-sqlite-1.1.7-1.2.1
babel-0.9.5-1.el5
python-babel-0.9.5-1.el5
python-pygments-1.1.1-1.el5

[bog@…]

Trac[SoC:formatter] ERROR: Macro WikiTicketCalendar(None) failed: 
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 710, in _macro_formatter
    return macro.process(args, in_paragraph=True)
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 334, in process
    text = "</p>%s<p>" % _markup_to_unicode(text)
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 82, in _markup_to_unicode
    markup = stream.render('xhtml', encoding=None, strip_whitespace=False)
  File "/usr/lib/python2.4/site-packages/genshi/core.py", line 183, in render
    return encode(generator, method=method, encoding=encoding, out=out)
  File "/usr/lib/python2.4/site-packages/genshi/output.py", line 57, in encode
    return _encode(''.join(list(iterator)))
  File "/usr/lib/python2.4/site-packages/genshi/output.py", line 371, in __call__
    yield _emit(kind, data, escape(data, quotes=False))
  File "/usr/lib/python2.4/site-packages/genshi/core.py", line 517, in escape
    return cls(text)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)

[bog@…]

httpd.conf

PythonOption TracLocale ru_RU.UTF8

[Steffen Hoffmann]

Looks like you've found another unicode-unsafe part, while I meant to have fixed all possibly variables that could need utf8 encoding lately.

There is certainly no template in WikiTicketCalendarMacro right now nor did it exist before, so the Genshi error wording is certainly misleading, if not plain wrong. But we'll see.

Well, would you verify and tell me the exact code version/revision you're applying, please?

[bog@…]

It works when i comment TracLocale.

Trac  	0.12
Babel 	0.9.5
Genshi 	0.6
mod_python 	3.2.8
Pygments 	1.1.1
pysqlite 	2.3.3
Python 	2.4.3 (#1, Sep 3 2009, 15:37:37) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)]
pytz 	2010h
setuptools 	0.6c5
SQLite 	3.3.6
Subversion 	1.6.12 (r955767)
jQuery:	1.4.2

wikiticketcalendarmacro-r8937.zip

[Steffen Hoffmann]

Revision indicates latest version, fine. Hm, problem comes with that locale. Tested locales (see screenshots on wiki page) i.e. didn't change the weekday column headers to something with non-ASCII chars. This might be the problem, but I'll have to test with your locale to be sure. Didn't try before, since currently there are no Russian translations for this plugin.

[Steffen Hoffmann]

Make this top priority to investigate and fix, as it prevents use in localized environments, and I really care for that in my own applications as well.

[anonymous]

I create ru translation, but this doesn't help

I have trac 0.12 with lighttpd installation. I have "bin-environment" => ("LC_TIME" => ru_RU.utf8") When I comment this setting all works fine. Problem is that in ru_RU weekdays is in non-ASCII

boris@bsavelev:~$ LC_TIME=ru date
Mon Dec  6 17:47:30 MSK 2010
boris@bsavelev:~$ LC_TIME=ru_RU.UTF-8 date
Пнд Дек  6 17:47:34 MSK 2010

mounth also is not-ASCII

[Boris Savelev]

here the patch

Index: wikiticketcalendar/macro.py
===================================================================
--- wikiticketcalendar/macro.py (revision 9588)
+++ wikiticketcalendar/macro.py (working copy)
@@ -404,11 +404,11 @@
         heading(align='center')
 
         for day in calendar.weekheader(2).split()[:-2]:
-            col = tag.th(day)
+            col = tag.th(to_unicode(day))
             col(class_='workday', scope='col')
             heading(col)
         for day in calendar.weekheader(2).split()[-2:]:
-            col = tag.th(day)
+            col = tag.th(to_unicode(day))
             col(class_='weekend', scope='col')
             heading(col)

[Boris Savelev]

#: wikiticketcalendar/macro.py:435                                                                                                          
#, python-format                                                                                                                            
msgid "Go to page %s"                                                                                                                       
msgstr "Переход на страницу %s"                                                                                                             
                                                                                                                                            
#: wikiticketcalendar/macro.py:442                                                                                                          
#, python-format                                                                                                                            
msgid "Create page %s"                                                                                                                      
msgstr "Создать страницу %s"

also apply ru.po please

[anonymous]

another unicode error: self.config.get('wikiticketcalendar'... return unicode object format_date expect str object

patch:

@@ -497,7 +497,7 @@
                                     continue
                             else:
                                 duedate = format_date(day_dt,
-                                                      self.due_field_fmt)
+                                                      str(self.due_field_fmt))
                                 if not due == duedate:
                                     continue

[Steffen Hoffmann]

Replying to anonymous:

I create ru translation, but this doesn't help

This is what I've been waiting for, thanks. I does help, since now there is something for testing with non-ASCII days and months name.

[Steffen Hoffmann]

Replying to anonymous:

another unicode error: self.config.get('wikiticketcalendar'... return unicode object format_date expect str object

Oh, never expected that, really curious to see, what's your value/setting there. Would you mind to share it?

[Boris Savelev]

Error: Macro WikiTicketCalendar(cdate=1, query=type=суд) failed

strftime() argument 1 must be str, not unicode

I receive this error then set

[wikiticketcalendar]
ticket.due_field.name = next_session
ticket.due_field.format = %d.%m.%y

then comment ticket.due_field.format all works fine

env

Trac 	0.12
Babel 	0.9.5
Docutils 	0.5
Genshi 	0.6
MySQL 	server: "5.0.51a-24+lenny4", client: "5.0.51a", thread-safe: 1
MySQLdb 	1.2.2
Pygments 	0.10
Python 	2.5.2 (r252:60911, Jan 24 2010, 17:44:40) [GCC 4.3.2]
pytz 	2008c
setuptools 	0.6c8
Subversion 	1.5.1 (r32289)
jQuery:	1.3.2

trac with fastcgi on lighttpd

env for fastcgi

TRAC_ENV=...
PYTHON_EGG_CACHE=...
LC_TIME=ru_RU.UTF-8

PS: I novice in python, are my patches correct?

[Steffen Hoffmann]

(In [9596]) WikiTicketCalendarMacro: Add more changes to handle unicode strings properly, refs #7698.

Further improvements to handle issues with non-ASCII day and month names, that appear i.e. when locale has been set to ru_RU.UTF8, and Russian translation came from Boris Savelev too. Bumping version on trunk branch.

[Steffen Hoffmann]

WikiTicketCalenderMacro v1.2.4dev with locale set to 'ru_RU.UTF-8'

[Steffen Hoffmann]

Nice, thanks again to Boris for taking the time to investigate this.

Beware: When testing, I recognized, that you need the locale in question as a pre-requisite to see anything. In fact all Trac pages disappeared after setting 'ru_RU.UTF-8' and before installing that locale at my test system.

But I just see, we're not finished yet, since there's come more ...

[Boris Savelev]

great! thanks a lot!

[Steffen Hoffmann]

I nearly forgot, what was the remaining issue, so this is more obvious now.

[Ryan J Ollos]

The plugin is deprecated. Feature requests should be directed to WikiCalendarMacro.