Modify

Opened 6 years ago

Closed 3 years ago

#3741 closed defect (fixed)

Tracforms can't handle mutated vowel

Reported by: didley@… Owned by: hasienda
Priority: high Component: TracFormsPlugin
Severity: major Keywords: unicode umlauts
Cc: Trac Release: 0.11

Description

Whe I using mutated vowels in a Form for example ä in Tracforms the message comes up

'ascii' codec can't encode character u'\xf6' in position 0: ordinal not in range(128)

Other plugins or wiki can handle it. Is it possible to fix it?

Attachments (0)

Change History (11)

comment:1 Changed 4 years ago by hasienda

  • Keywords umlauts added

#5543 mentioned especially German umlauts and has been closed as a duplicate of this ticket.

comment:2 Changed 4 years ago by hasienda

  • Keywords unicode added
  • Priority changed from normal to high
  • Severity changed from normal to major

Confirmed, working towards a solution, since I require it too, and that can't be that hard. It might just be about missing unicode encoding for form content in some places.

comment:3 Changed 4 years ago by rjollos

I'll watch for you patch since this is a frequent problem with plugins and I need to wrap my head around how to fix these issues.

comment:4 Changed 4 years ago by hasienda

  • Owner changed from rharkins to hasienda

Sure, this is one of my short-term tasks. Thanks for taking care.

comment:5 Changed 3 years ago by hasienda

  • Status changed from new to assigned

It took me a lot more time for code studies than expected initially. While the code looks really clean, it's not easy to get all details, since it heavily uses modularization and recursions.

comment:6 Changed 3 years ago by hasienda

(In [9931]) TracFormsPlugin: Allow non-ASCII characters in text input and other fields, refs #3741.

X(HT)ML conform unicode character escaping has been largely inspired by

http://www.xml.com/pub/a/2005/06/15/py-xml.html

comment:7 Changed 3 years ago by hasienda

(In [9967]) TracFormsPlugin: Move XML unicode handling into dedicated script, refs #3741.

Minor code cleanup and preparation for ongoing development started as well.

comment:8 follow-up: Changed 3 years ago by didley@…

Now it works for me. Thanx a lot.

didley

comment:9 in reply to: ↑ 8 Changed 3 years ago by hasienda

Replying to didley@gmx.de:

Now it works for me. Thanx a lot.

Glad to hear that. But as the values are stored unchanged with the numeric unicode-char-escapes I'm still not totally satisfied by this solution. Hence the still open ticket.

It's just fair to tell you, that the case is not fully resolved from my point of view. Especially regarding #3500 I do suspect, that searching a db full of such escaped strings is not as accessible performance-wise as plain strings. So with the last commit I've already prepared to reverse the escaping process by a profiled algorithm selected from a choice of three. Be prepared to do some db cleanup in case I'll implement this for the storage backend. In fact I've postponed my own production roll-out until I've done the search integration (hopefully by next week).

OTOH it might be possible to escape search strings for the Forms realm as well to obsolete the unescaping process. What would be really needed then, is to provide an abstraction layer, that does handle access to TracForms data in db for all access (i.e. from other plugins too).

comment:10 Changed 3 years ago by hasienda

By now it should be quite safe to follow with at least trunk revision [10005], since I'm using that code in production now.

Since I've switched user input handling to use unicode encoding by default, there are few places to worry about, i.e. non-ASCII variables and usernames, but this is not such a big restriction anymore, if it is a real issue at all. Nevertheless I'm looking forwards to a release candidate for a formal 0.3 release, maybe in April.

Once again, German umlauts work flawlessly, and anyone in need of using similar non-ASCII chars should follow and report his/her experience, please. Thanks for taking care. I'll close this ticket in a while, if there's no more complaint related to the topic.

comment:11 Changed 3 years ago by hasienda

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [10143]) TracFormsPlugin: Releasing version 0.3, pushing development to 0.4, closes #3445, #3550, #3741, #4759 and #8258, refs #3388 and #6993.

This is a major release requiring a Trac environment upgrade.

While the parser logic remains unchanged, there is a lot new supplementary
funcionality to make TracForms behave more like the existing Trac core
resources (ticket, wiki, attachment, ...).

This version performs a series of non-trivial db schema changes, that
especially may leave traces of stale forms (i.e. recorded for wiki pages,
that don't exist anymore). So please make sure to read the changelog,
BACKUP your environment(s) before installing this version as usual and check
the new db tables forms, forms_fields and forms_history after upgrading.

Add Comment

Modify Ticket

Action
as closed .
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from hasienda. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.