Modify

Opened 14 years ago

Closed 7 years ago

#7698 closed defect (wontfix)

Handle unicode arguments for embedded custom TracQuery too

Reported by: bog@… Owned by: Steffen Hoffmann
Priority: high Component: WikiTicketCalendarMacro
Severity: major Keywords: unicode 0.11 0.12
Cc: Ryan J Ollos, Boris Savelev Trac Release: 0.12

Description

I had tried 0.11 (as single file) and 0.12 (as egg install in .../plugins) and 0.12 (as easy_install). In any case i have:

Genshi UnicodeDecodeError error while rendering template (unknown template location)

CentOS 5.4 + http://packages.sw.be/trac/
Apache 2.2.3
trac 0.12
python-setuptools-0.6c5-2.el5
python-genshi-0.6-1.el5.rf
python-sqlite-1.1.7-1.2.1
babel-0.9.5-1.el5
python-babel-0.9.5-1.el5
python-pygments-1.1.1-1.el5

Attachments (1)

20101206_wikiticketcalendar_ru.png (24.3 KB) - added by Steffen Hoffmann 13 years ago.
WikiTicketCalenderMacro v1.2.4dev with locale set to 'ru_RU.UTF-8'

Download all attachments as: .zip

Change History (21)

comment:1 Changed 13 years ago by bog@…

Trac[SoC:formatter] ERROR: Macro WikiTicketCalendar(None) failed: 
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 710, in _macro_formatter
    return macro.process(args, in_paragraph=True)
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 334, in process
    text = "</p>%s<p>" % _markup_to_unicode(text)
  File "/usr/lib/python2.4/site-packages/trac/wiki/formatter.py", line 82, in _markup_to_unicode
    markup = stream.render('xhtml', encoding=None, strip_whitespace=False)
  File "/usr/lib/python2.4/site-packages/genshi/core.py", line 183, in render
    return encode(generator, method=method, encoding=encoding, out=out)
  File "/usr/lib/python2.4/site-packages/genshi/output.py", line 57, in encode
    return _encode(''.join(list(iterator)))
  File "/usr/lib/python2.4/site-packages/genshi/output.py", line 371, in __call__
    yield _emit(kind, data, escape(data, quotes=False))
  File "/usr/lib/python2.4/site-packages/genshi/core.py", line 517, in escape
    return cls(text)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)

comment:2 Changed 13 years ago by bog@…

httpd.conf

PythonOption TracLocale ru_RU.UTF8

comment:3 Changed 13 years ago by Steffen Hoffmann

Cc: Ryan J Ollos added; anonymous removed
Keywords: unicode 0.11 added
Summary: Genshi UnicodeDecodeError error while rendering template (unknown template location)Genshi UnicodeDecodeError while rendering

Looks like you've found another unicode-unsafe part, while I meant to have fixed all possibly variables that could need utf8 encoding lately.

There is certainly no template in WikiTicketCalendarMacro right now nor did it exist before, so the Genshi error wording is certainly misleading, if not plain wrong. But we'll see.

Well, would you verify and tell me the exact code version/revision you're applying, please?

comment:4 Changed 13 years ago by bog@…

It works when i comment TracLocale.

Trac  	0.12
Babel 	0.9.5
Genshi 	0.6
mod_python 	3.2.8
Pygments 	1.1.1
pysqlite 	2.3.3
Python 	2.4.3 (#1, Sep 3 2009, 15:37:37) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)]
pytz 	2010h
setuptools 	0.6c5
SQLite 	3.3.6
Subversion 	1.6.12 (r955767)
jQuery:	1.4.2

wikiticketcalendarmacro-r8937.zip

comment:5 Changed 13 years ago by Steffen Hoffmann

Revision indicates latest version, fine. Hm, problem comes with that locale. Tested locales (see screenshots on wiki page) i.e. didn't change the weekday column headers to something with non-ASCII chars. This might be the problem, but I'll have to test with your locale to be sure. Didn't try before, since currently there are no Russian translations for this plugin.

comment:6 Changed 13 years ago by Steffen Hoffmann

Priority: normalhigh

Make this top priority to investigate and fix, as it prevents use in localized environments, and I really care for that in my own applications as well.

comment:7 Changed 13 years ago by anonymous

Cc: Boris Savelev added
Keywords: 0.12 added

I create ru translation, but this doesn't help

I have trac 0.12 with lighttpd installation. I have "bin-environment" => ("LC_TIME" => ru_RU.utf8") When I comment this setting all works fine. Problem is that in ru_RU weekdays is in non-ASCII

boris@bsavelev:~$ LC_TIME=ru date
Mon Dec  6 17:47:30 MSK 2010
boris@bsavelev:~$ LC_TIME=ru_RU.UTF-8 date
Пнд Дек  6 17:47:34 MSK 2010

mounth also is not-ASCII

comment:8 Changed 13 years ago by Boris Savelev

here the patch

Index: wikiticketcalendar/macro.py
===================================================================
--- wikiticketcalendar/macro.py (revision 9588)
+++ wikiticketcalendar/macro.py (working copy)
@@ -404,11 +404,11 @@
         heading(align='center')
 
         for day in calendar.weekheader(2).split()[:-2]:
-            col = tag.th(day)
+            col = tag.th(to_unicode(day))
             col(class_='workday', scope='col')
             heading(col)
         for day in calendar.weekheader(2).split()[-2:]:
-            col = tag.th(day)
+            col = tag.th(to_unicode(day))
             col(class_='weekend', scope='col')
             heading(col)

comment:9 Changed 13 years ago by Boris Savelev

#: wikiticketcalendar/macro.py:435                                                                                                          
#, python-format                                                                                                                            
msgid "Go to page %s"                                                                                                                       
msgstr "Переход на страницу %s"                                                                                                             
                                                                                                                                            
#: wikiticketcalendar/macro.py:442                                                                                                          
#, python-format                                                                                                                            
msgid "Create page %s"                                                                                                                      
msgstr "Создать страницу %s"

also apply ru.po please

comment:10 Changed 13 years ago by anonymous

another unicode error: self.config.get('wikiticketcalendar'... return unicode object format_date expect str object

patch:

@@ -497,7 +497,7 @@
                                     continue
                             else:
                                 duedate = format_date(day_dt,
-                                                      self.due_field_fmt)
+                                                      str(self.due_field_fmt))
                                 if not due == duedate:
                                     continue

comment:11 in reply to:  7 Changed 13 years ago by Steffen Hoffmann

Status: newassigned

Replying to anonymous:

I create ru translation, but this doesn't help

This is what I've been waiting for, thanks. I does help, since now there is something for testing with non-ASCII days and months name.

comment:12 in reply to:  10 Changed 13 years ago by Steffen Hoffmann

Replying to anonymous:

another unicode error: self.config.get('wikiticketcalendar'... return unicode object format_date expect str object

Oh, never expected that, really curious to see, what's your value/setting there. Would you mind to share it?

comment:13 Changed 13 years ago by Boris Savelev

Resolution: fixed
Status: assignedclosed
Error: Macro WikiTicketCalendar(cdate=1, query=type=суд) failed

strftime() argument 1 must be str, not unicode

I receive this error then set

[wikiticketcalendar]
ticket.due_field.name = next_session
ticket.due_field.format = %d.%m.%y

then comment ticket.due_field.format all works fine

env

Trac 	0.12
Babel 	0.9.5
Docutils 	0.5
Genshi 	0.6
MySQL 	server: "5.0.51a-24+lenny4", client: "5.0.51a", thread-safe: 1
MySQLdb 	1.2.2
Pygments 	0.10
Python 	2.5.2 (r252:60911, Jan 24 2010, 17:44:40) [GCC 4.3.2]
pytz 	2008c
setuptools 	0.6c8
Subversion 	1.5.1 (r32289)
jQuery:	1.3.2

trac with fastcgi on lighttpd

env for fastcgi

TRAC_ENV=...
PYTHON_EGG_CACHE=...
LC_TIME=ru_RU.UTF-8

PS: I novice in python, are my patches correct?

comment:14 Changed 13 years ago by Boris Savelev

Resolution: fixed
Status: closedreopened

comment:15 Changed 13 years ago by Boris Savelev

Status: reopenednew

comment:16 Changed 13 years ago by Steffen Hoffmann

(In [9596]) WikiTicketCalendarMacro: Add more changes to handle unicode strings properly, refs #7698.

Further improvements to handle issues with non-ASCII day and month names, that appear i.e. when locale has been set to ru_RU.UTF8, and Russian translation came from Boris Savelev too. Bumping version on trunk branch.

Changed 13 years ago by Steffen Hoffmann

WikiTicketCalenderMacro v1.2.4dev with locale set to 'ru_RU.UTF-8'

comment:17 Changed 13 years ago by Steffen Hoffmann

Status: newassigned

WikiTicketCalenderMacro v1.2.4dev with locale set to 'ru_RU.UTF-8'

Nice, thanks again to Boris for taking the time to investigate this.

Beware: When testing, I recognized, that you need the locale in question as a pre-requisite to see anything. In fact all Trac pages disappeared after setting 'ru_RU.UTF-8' and before installing that locale at my test system.

But I just see, we're not finished yet, since there's come more ...

comment:18 Changed 13 years ago by Boris Savelev

great! thanks a lot!

comment:19 Changed 13 years ago by Steffen Hoffmann

Summary: Genshi UnicodeDecodeError while renderingHandle unicode arguments for embedded custom TracQuery too

I nearly forgot, what was the remaining issue, so this is more obvious now.

comment:20 Changed 7 years ago by Ryan J Ollos

Resolution: wontfix
Status: assignedclosed

The plugin is deprecated. Feature requests should be directed to WikiCalendarMacro.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Steffen Hoffmann.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.