Opened 10 years ago

Closed 4 years ago

# ticket.get appears to die on ticket custom fields

Reported by: Owned by: greg@… osimons normal XmlRpcPlugin blocker thijs 0.10

### Description

When trying to use ticket.get() against my trac installation I get a TypeError exception from dump_struct in xmlrpclib.py due to the type of one of my custom fields not being string. They are, in fact type 'unicode' so I'm not really sure if they are wrong or the plugin is wrong but either way it doesn't seem to work. This was also posted to the trac mailing list.

Sorry to respond to my own post but I've been debugging this a bit and
have more of an idea of what is going on.

I played around with the exception handling in xmlrpclib.py. It turns
out the TypeError exception gets thrown on one of my custom ticket
fields. The field is defined in trac.ini as:

verified = checkbox
verified.label = Fix Verified
verified.value = 0
verified.order = 7

and the type is somehow unicode. I have a few other checkbox custom
fields and if they are processed in order then they are all fine. This
is the same field I had problems with in ticket #3939 so now I'm
wondering if there is just something dodgy with that field.

Anyone have an idea of how to track this down?

Thanks,
Greg

On Sat, 2006-11-04 at 18:23 -0500, Greg Dickie wrote:
> Hi,
>
>   Sorry in advance if this is a FAQ but I'm an xmlrpc virgin ;-). I want
> to use xmlrpc to create tickets a part of a merge review process but I
> can't even get the examples to work right ;-). This is trac 0.10 and
> using the example:
>
> import xmlrpclib
>
> server = xmlrpclib.ServerProxy("http://trac/Sledgehammer/xmlrpc")
>
> multicall = xmlrpclib.MultiCall(server)
> for ticket in server.ticket.query("owner=greg"):
>     multicall.ticket.get(ticket)
> print map(str, multicall())
>
>
> gives this:
>
> Traceback (most recent call last):
>   File "tt", line 8, in ?
>     print map(str, multicall())
>   File "/usr/lib/python2.4/xmlrpclib.py", line 945, in __call__
>     return
> MultiCallIterator(self.__server.system.multicall(marshalled_list))
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
>     return self.__send(self.__name, args)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
>     verbose=self.__verbose
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
>     return self._parse_response(h.getfile(), sock)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
>     return u.close()
>   File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
>     raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 2: "'dictionary key must be string' while
> executing 'system.multicall()'">
>
>
> which seems to be to be a problem in the data being received.
>
> Anyone know what I'm doing wrong?
>
> Thanks,
> Greg
>



Thanks, Greg

### comment:1 Changed 10 years ago by anonymous

• Severity changed from normal to blocker

### comment:2 Changed 10 years ago by gotoh

I think you'd better to check trac.log to get detail of error ('dictionary key must be string') beacuse error is occured while executing on trac.

I tried your code but I cannot reproduced in my environment which has selection custom field.

### comment:3 Changed 10 years ago by anonymous

Thanks for the suggestion. Note the traceback is slightly different because I changed the exception to tell me which field had the problem but its the same exception. From trac.log on the server I get:

2006-11-21 07:09:28,115 Trac[web_ui] ERROR: Traceback (most recent call last):
File "build/bdist.linux-i686/egg/tracrpc/web_ui.py", line 58, in process_reque
st
File "/usr/lib/python2.3/xmlrpclib.py", line 969, in dumps
data = m.dumps(params)
File "/usr/lib/python2.3/xmlrpclib.py", line 604, in dumps
dump(v, write)
File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
f(self, value, write)
File "/usr/lib/python2.3/xmlrpclib.py", line 676, in dump_array
dump(v, write)
File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
f(self, value, write)
File "/usr/lib/python2.3/xmlrpclib.py", line 699, in dump_struct
raise NameError,  k
NameError: verified


### comment:4 Changed 10 years ago by gotoh

Yet I cannot reproduce your error although I've added same custom field 'verified' on trac 0.10.2 (on windows and python 2.3.5). I prefer to check (by logging, etc.) type of object in variable 'k' to ensure it is string or not when trapped before raise NameError (by your debug code, I guess) becaues the error "dictionary key must be string" is raised when 'k' is nether StringType nor UnicodeType (in xmlrpclib.dump_struct()).

### comment:5 follow-up: ↓ 7 Changed 10 years ago by anonymous

Thank you for looking at this. I have verified the type and it is, in fact, unicode but xmlrpclib only checks for string, if it check for unicode it would be fine. See below (with my debug code in it):

    def dump_struct(self, value, write, escape=escape):
i = id(value)
if self.memo.has_key(i):
raise TypeError, "cannot marshal recursive dictionaries"
self.memo[i] = None
dump = self.__dump
write("<value><struct>\n")
for k in value.keys():
write("<member>\n")
try:
raise NameError,  type(k)
except NameError:
print k
if type(k) is not StringType:
try:
raise TypeError, "dictionary key must be string"
except TypeError:
raise NameError,  k
raise
write("<name>%s</name>\n" % escape(k))
dump(value[k], write)
write("</member>\n")
write("</struct></value>\n")
del self.memo[i]
dispatch[DictType] = dump_struct


This is python 2.3.4-14.3 on CentOS 4U4

### comment:6 Changed 10 years ago by anonymous

Just for fun, I added a check in for UnicodeType in xmlrpclib.py and now it seems to work. So I suppose the bug is really in xmlrpclib but any idea why custom fields are unicode but regular fields are string?

### comment:7 in reply to: ↑ 5 Changed 10 years ago by gotoh

This is python 2.3.4-14.3 on CentOS 4U4

dump_struct() of python 2.3.5 is:

    def dump_struct(self, value, write, escape=escape):
i = id(value)
if self.memo.has_key(i):
raise TypeError, "cannot marshal recursive dictionaries"
self.memo[i] = None
dump = self.__dump
write("<value><struct>\n")
for k, v in value.items():
write("<member>\n")
if type(k) is not StringType:
if unicode and type(k) is UnicodeType:
k = k.encode(self.encoding)
else:
raise TypeError, "dictionary key must be string"
write("<name>%s</name>\n" % escape(k))
dump(v, write)
write("</member>\n")
write("</struct></value>\n")
del self.memo[i]


It seems there is a difference in unicode consideration between 2.3.4 and 2.3.5. Thus, easiest workaround is modifying xmlrpclib.py like above or upgrade to 2.3.5 if possible. It seems a bug of xmlrpclib.

The reason of name of non-custom fields are not unicode is field name literals in TicketSystem.get_ticket_fields() are not unicode string. On the other hand, strings read from trac.ini are all unicode. It is the difference, I guess.

Following patch might be help if we make workaround in xmlrpcplugin to support python 2.3.4 or erlier.

• ## ticket.py

 def get(self, req, id): """ Fetch a ticket. Returns [id, time_created, time_changed, attributes]. """ t = model.Ticket(self.env, id) return (t.id, t.time_created, t.time_changed, t.values) values = {} for k, v in t.values.items(): import sys if sys.version_info[:3] < (2,3,5) and isinstance(k, unicode): k = k.encode('utf-8') # workaround for old xmlrpclib values[k] = v return (t.id, t.time_created, t.time_changed, values) def create(self, req, summary, description, attributes = {}): """ Create a new ticket, returning the ticket ID. """

### comment:8 follow-up: ↓ 9 Changed 10 years ago by greg@…

Indeed it seems to be a problem in the python distribution however since I guess alot of people will want to stay with their distribution's currently "blessed" version its probably better to fix the plugin code. Your patch above is great but that is only for the get method. Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

Thanks again for looking at this.

### comment:9 in reply to: ↑ 8 Changed 10 years ago by gotoh

Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

I cannot test to find good or not because I don't have 2.3.4 environment (windows and debian). Please test for other methods on your environment then report here.

### comment:10 Changed 10 years ago by rob.sharp@…

I have patched the plugin as mentioned above, looks like it works well!

### comment:11 Changed 7 years ago by osimons

#1718 closed as duplicate.

### comment:12 Changed 6 years ago by thijs

• Cc thijs added; anonymous removed
• Owner changed from athomas to osimons

python 2.3 is very old and proably shouldn't be supported anymore?

### comment:13 Changed 4 years ago by osimons

• Resolution set to worksforme
• Status changed from new to closed

Problem is way too old... Whatever the problem was it is working now with newer versions. Just upgrade everything.