Umlauts doesn't work

[falkb]

In function def execute_query(env, req, query_args): of utils.py, I tried to add

estimationtoolsplugin/trunk/estimationtools/utils.py

@@ -134 +137 @@
                         .replace('+', ' ')\
                         .replace('%23', '#')\
                         .replace('%28', '(')\
-                        .replace('%29', ')')
+                        .replace('%29', ')')\
+                        .replace('%C3%84', 'Ä')\
+                        .replace('%C3%96', 'Ö')\
+                        .replace('%C3%9C', 'Ü')\
+                        .replace('%C3%A4', 'ä')\
+                        .replace('%C3%B6', 'ö')\
+                        .replace('%C3%BC', 'ü')\
+                        .replace('%C3%9F', 'ß')
     env.log.debug("query_string: %s" % query_string)
     query = Query.from_string(env, query_string)

but just get this error:

Error: Macro WorkloadChart(milestone=testgehäuse) failed

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

How must umlauts be fixed?

[anonymous]

It's still not in the trunk, why?

[Jun Omae]

That patch cannot fix this issue. The same issue would be raised if other unicode characters are used.

Could you please try the following patch?

estimationtoolsplugin/trunk/estimationtools/utils.py

@@ -14 +14 @@
 from trac.config import Option, ListOption, BoolOption
 from trac.core import TracError, Component, implements
 from trac.ticket.query import Query
-from trac.util.datefmt import from_utimestamp, utc
-from trac.util.text import unicode_urlencode
+from trac.util.datefmt import from_utimestamp
 from trac.web.api import IRequestHandler, RequestDone
 from trac.wiki.api import parse_args
 
@@ -147 +146 @@
 def execute_query(env, req, query_args):
     # set maximum number of returned tickets to 0 to get all tickets at once
     query_args['max'] = 0
-    # urlencode the args, converting back a few vital exceptions:
-    # see the authorized fields in the query language in
-    # http://trac.edgewall.org/wiki/TracQuery#QueryLanguage
-    query_string = unicode_urlencode(query_args).replace('%21=', '!=') \
-        .replace('%21%7E=', '!~=') \
-        .replace('%7E=', '~=') \
-        .replace('%5E=', '^=') \
-        .replace('%24=', '$=') \
-        .replace('%21%5E=', '!^=') \
-        .replace('%21%24=', '!$=') \
-        .replace('%7C', '|') \
-        .replace('+', ' ') \
-        .replace('%23', '#') \
-        .replace('%28', '(') \
-        .replace('%29', ')')
+    query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
     env.log.debug("query_string: %s", query_string)
     query = Query.from_string(env, query_string)
 

[falkb]

Jun, your patch works well. Thanks!

[anonymous]

Could you commit the patch, please! And close this ticket then.

[falkb]

I tried to commit the patch above but have no permissions. So I'm giving up with letting you know this could have been fixed. Neither can I set Jun Omae on CC of this ticket.

[Ryan J Ollos]

In 16736:

EstimationTools 0.5.0dev: Fix incorrect encoding in query

Patch by Jun Omae.

Fixes #10135.

[Ryan J Ollos]

One test failure:

$python -m unittest estimationtools.tests.burndownchart.BurndownChartTestCase.test_url_encode
E
======================================================================
ERROR: test_url_encode (estimationtools.tests.burndownchart.BurndownChartTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "estimationtools/tests/burndownchart.py", line 246, in test_url_encode
    % (start, end))
  File "estimationtools/burndownchart.py", line 90, in expand_macro
    timetable = self._calculate_timetable(options, query_args, req)
  File "estimationtools/burndownchart.py", line 189, in _calculate_timetable
    tickets = execute_query(self.env, req, query_args)
  File "estimationtools/utils.py", line 151, in execute_query
    query = Query.from_string(env, query_string)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/query.py", line 170, in from_string
    raise QuerySyntaxError(_('Query filter requires field and '
QuerySyntaxError: Query filter requires field and constraints separated by a "="

----------------------------------------------------------------------
Ran 1 test in 0.022s

FAILED (errors=1)

query_args is:

{'max': 0, u'hours_remaining!': None, 'milestone': 'One & Two'}

query_string is:

max=0&hours_remaining!=None&milestone=One & Two

Issue seems to be associated with estimationtoolsplugin/trunk/estimationtools/burndownchart.py@16592:188#L177​.

[Jun Omae]

Hmm. That patch is from ​trac:source:/tags/trac-1.0.15/trac/ticket/query.py@:1371-1372#L1366.

It seems meta characters &=% should be quoted.

estimationtoolsplugin/trunk/estimationtools/utils.py

@@ -145 +145 @@
 
 def execute_query(env, req, query_args):
     # set maximum number of returned tickets to 0 to get all tickets at once
+    def quote(v):
+        return unicode(v).replace('%', '%25').replace('&', '%26') \
+                         .replace('=', '%3D')
+
     query_args['max'] = 0
-    query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
+    query_string = '&'.join('%s=%s' % (quote(item[0]), quote(item[1]))
+                            for item in query_args.iteritems())
     env.log.debug("query_string: %s", query_string)
     query = Query.from_string(env, query_string)
 

[Ryan J Ollos]

Thanks, test passing now.

[Ryan J Ollos]

In 16740:

TracEstimationTools 0.5.0dev: Quote meta characters

Fixes #10135.