Modify

Opened 8 years ago

Closed 2 years ago

#871 closed defect (worksforme)

ticket.get appears to die on ticket custom fields

Reported by: greg@… Owned by: osimons
Priority: normal Component: XmlRpcPlugin
Severity: blocker Keywords:
Cc: thijs Trac Release: 0.10

Description

When trying to use ticket.get() against my trac installation I get a TypeError exception from dump_struct in xmlrpclib.py due to the type of one of my custom fields not being string. They are, in fact type 'unicode' so I'm not really sure if they are wrong or the plugin is wrong but either way it doesn't seem to work. This was also posted to the trac mailing list.

Sorry to respond to my own post but I've been debugging this a bit and
have more of an idea of what is going on.

I played around with the exception handling in xmlrpclib.py. It turns
out the TypeError exception gets thrown on one of my custom ticket
fields. The field is defined in trac.ini as:

verified = checkbox
verified.label = Fix Verified
verified.value = 0
verified.order = 7

and the type is somehow unicode. I have a few other checkbox custom
fields and if they are processed in order then they are all fine. This
is the same field I had problems with in ticket #3939 so now I'm
wondering if there is just something dodgy with that field. 

Anyone have an idea of how to track this down?

Thanks,
Greg


On Sat, 2006-11-04 at 18:23 -0500, Greg Dickie wrote:
> Hi,
> 
>   Sorry in advance if this is a FAQ but I'm an xmlrpc virgin ;-). I want
> to use xmlrpc to create tickets a part of a merge review process but I
> can't even get the examples to work right ;-). This is trac 0.10 and
> using the example:
> 
> import xmlrpclib
> 
> server = xmlrpclib.ServerProxy("http://trac/Sledgehammer/xmlrpc")
> 
> multicall = xmlrpclib.MultiCall(server)
> for ticket in server.ticket.query("owner=greg"):
>     multicall.ticket.get(ticket)
> print map(str, multicall())
> 
> 
> gives this:
> 
> Traceback (most recent call last):
>   File "tt", line 8, in ?
>     print map(str, multicall())
>   File "/usr/lib/python2.4/xmlrpclib.py", line 945, in __call__
>     return
> MultiCallIterator(self.__server.system.multicall(marshalled_list))
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
>     return self.__send(self.__name, args)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
>     verbose=self.__verbose
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
>     return self._parse_response(h.getfile(), sock)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
>     return u.close()
>   File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
>     raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 2: "'dictionary key must be string' while
> executing 'system.multicall()'">
> 
> 
> which seems to be to be a problem in the data being received.
> 
> Anyone know what I'm doing wrong?
> 
> Thanks,
> Greg
> 

Thanks,
Greg

Attachments (0)

Change History (13)

comment:1 Changed 8 years ago by anonymous

  • Severity changed from normal to blocker

comment:2 Changed 8 years ago by gotoh

I think you'd better to check trac.log to get detail of error ('dictionary key must be string')
beacuse error is occured while executing on trac.

I tried your code but
I cannot reproduced in my environment which has selection custom field.

comment:3 Changed 8 years ago by anonymous

Thanks for the suggestion. Note the traceback is slightly different because I changed the exception to tell me which field had the problem but its the same exception. From trac.log on the server I get:

2006-11-21 07:09:28,115 Trac[web_ui] ERROR: Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/tracrpc/web_ui.py", line 58, in process_reque
st
  File "/usr/lib/python2.3/xmlrpclib.py", line 969, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.3/xmlrpclib.py", line 604, in dumps
    dump(v, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
    f(self, value, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 676, in dump_array
    dump(v, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
    f(self, value, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 699, in dump_struct
    raise NameError,  k
NameError: verified

comment:4 Changed 8 years ago by gotoh

Yet I cannot reproduce your error although I've added same custom field 'verified' on trac 0.10.2
(on windows and python 2.3.5).
I prefer to check (by logging, etc.) type of object in variable 'k' to ensure it is string or not
when trapped before raise NameError (by your debug code, I guess)
becaues the error "dictionary key must be string" is raised when 'k'
is nether StringType nor UnicodeType (in xmlrpclib.dump_struct()).

comment:5 follow-up: Changed 8 years ago by anonymous

Thank you for looking at this. I have verified the type and it is, in fact, unicode but xmlrpclib only checks for string, if it check for unicode it would be fine. See below (with my debug code in it):

    def dump_struct(self, value, write, escape=escape):
        i = id(value)
        if self.memo.has_key(i):
            raise TypeError, "cannot marshal recursive dictionaries"
        self.memo[i] = None
        dump = self.__dump
        write("<value><struct>\n")
        for k in value.keys():
            write("<member>\n")
            try:
              raise NameError,  type(k)
            except NameError:
              print k
            if type(k) is not StringType:
              try:
                  raise TypeError, "dictionary key must be string"
              except TypeError:
                raise NameError,  k
                raise
            write("<name>%s</name>\n" % escape(k))
            dump(value[k], write)
            write("</member>\n")
        write("</struct></value>\n")
        del self.memo[i]
    dispatch[DictType] = dump_struct

This is python 2.3.4-14.3 on CentOS 4U4

comment:6 Changed 8 years ago by anonymous

Just for fun, I added a check in for UnicodeType in xmlrpclib.py and now it seems to work. So I suppose the bug is really in xmlrpclib but any idea why custom fields are unicode but regular fields are string?

comment:7 in reply to: ↑ 5 Changed 8 years ago by gotoh

Replying to anonymous:

This is python 2.3.4-14.3 on CentOS 4U4

dump_struct() of python 2.3.5 is:

    def dump_struct(self, value, write, escape=escape):
        i = id(value)
        if self.memo.has_key(i):
            raise TypeError, "cannot marshal recursive dictionaries"
        self.memo[i] = None
        dump = self.__dump
        write("<value><struct>\n")
        for k, v in value.items():
            write("<member>\n")
            if type(k) is not StringType:
                if unicode and type(k) is UnicodeType:
                    k = k.encode(self.encoding)
                else:
                    raise TypeError, "dictionary key must be string"
            write("<name>%s</name>\n" % escape(k))
            dump(v, write)
            write("</member>\n")
        write("</struct></value>\n")
        del self.memo[i]

It seems there is a difference in unicode consideration between 2.3.4 and 2.3.5.
Thus, easiest workaround is modifying xmlrpclib.py like above or upgrade to 2.3.5 if possible.
It seems a bug of xmlrpclib.

The reason of name of non-custom fields are not unicode is
field name literals in TicketSystem.get_ticket_fields() are not unicode string.
On the other hand, strings read from trac.ini are all unicode.
It is the difference, I guess.

Following patch might be help if we make workaround in xmlrpcplugin to
support python 2.3.4 or erlier.

  • ticket.py

     
    6767    def get(self, req, id): 
    6868        """ Fetch a ticket. Returns [id, time_created, time_changed, attributes]. """ 
    6969        t = model.Ticket(self.env, id) 
    70         return (t.id, t.time_created, t.time_changed, t.values) 
     70        values = {} 
     71        for k, v in t.values.items(): 
     72            import sys 
     73            if sys.version_info[:3] < (2,3,5) and isinstance(k, unicode): 
     74                k = k.encode('utf-8') # workaround for old xmlrpclib 
     75            values[k] = v 
     76        return (t.id, t.time_created, t.time_changed, values) 
    7177 
    7278    def create(self, req, summary, description, attributes = {}): 
    7379        """ Create a new ticket, returning the ticket ID. """ 

comment:8 follow-up: Changed 8 years ago by greg@…

Indeed it seems to be a problem in the python distribution however since I guess alot of people will want to stay with their distribution's currently "blessed" version its probably better to fix the plugin code. Your patch above is great but that is only for the get method. Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

Thanks again for looking at this.

comment:9 in reply to: ↑ 8 Changed 8 years ago by gotoh

Replying to greg@max-t.com:

Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

I cannot test to find good or not because I don't have 2.3.4 environment (windows and debian).
Please test for other methods on your environment then report here.

comment:10 Changed 8 years ago by rob.sharp@…

I have patched the plugin as mentioned above, looks like it works well!

comment:11 Changed 5 years ago by osimons

#1718 closed as duplicate.

comment:12 Changed 4 years ago by thijs

  • Cc thijs added
  • Owner changed from athomas to osimons

python 2.3 is very old and proably shouldn't be supported anymore?

comment:13 Changed 2 years ago by osimons

  • Resolution set to worksforme
  • Status changed from new to closed

Problem is way too old... Whatever the problem was it is working now with newer versions. Just upgrade everything.

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.