Modify

Opened 18 years ago

Closed 12 years ago

#871 closed defect (worksforme)

ticket.get appears to die on ticket custom fields

Reported by: greg@… Owned by: osimons
Priority: normal Component: XmlRpcPlugin
Severity: blocker Keywords:
Cc: Thijs Triemstra Trac Release: 0.10

Description

When trying to use ticket.get() against my trac installation I get a TypeError exception from dump_struct in xmlrpclib.py due to the type of one of my custom fields not being string. They are, in fact type 'unicode' so I'm not really sure if they are wrong or the plugin is wrong but either way it doesn't seem to work. This was also posted to the trac mailing list.

Sorry to respond to my own post but I've been debugging this a bit and
have more of an idea of what is going on.

I played around with the exception handling in xmlrpclib.py. It turns
out the TypeError exception gets thrown on one of my custom ticket
fields. The field is defined in trac.ini as:

verified = checkbox
verified.label = Fix Verified
verified.value = 0
verified.order = 7

and the type is somehow unicode. I have a few other checkbox custom
fields and if they are processed in order then they are all fine. This
is the same field I had problems with in ticket #3939 so now I'm
wondering if there is just something dodgy with that field. 

Anyone have an idea of how to track this down?

Thanks,
Greg


On Sat, 2006-11-04 at 18:23 -0500, Greg Dickie wrote:
> Hi,
> 
>   Sorry in advance if this is a FAQ but I'm an xmlrpc virgin ;-). I want
> to use xmlrpc to create tickets a part of a merge review process but I
> can't even get the examples to work right ;-). This is trac 0.10 and
> using the example:
> 
> import xmlrpclib
> 
> server = xmlrpclib.ServerProxy("http://trac/Sledgehammer/xmlrpc")
> 
> multicall = xmlrpclib.MultiCall(server)
> for ticket in server.ticket.query("owner=greg"):
>     multicall.ticket.get(ticket)
> print map(str, multicall())
> 
> 
> gives this:
> 
> Traceback (most recent call last):
>   File "tt", line 8, in ?
>     print map(str, multicall())
>   File "/usr/lib/python2.4/xmlrpclib.py", line 945, in __call__
>     return
> MultiCallIterator(self.__server.system.multicall(marshalled_list))
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
>     return self.__send(self.__name, args)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
>     verbose=self.__verbose
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
>     return self._parse_response(h.getfile(), sock)
>   File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
>     return u.close()
>   File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
>     raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 2: "'dictionary key must be string' while
> executing 'system.multicall()'">
> 
> 
> which seems to be to be a problem in the data being received.
> 
> Anyone know what I'm doing wrong?
> 
> Thanks,
> Greg
> 

Thanks, Greg

Attachments (0)

Change History (13)

comment:1 Changed 18 years ago by anonymous

Severity: normalblocker

comment:2 Changed 18 years ago by Shun-ichi Goto

I think you'd better to check trac.log to get detail of error ('dictionary key must be string') beacuse error is occured while executing on trac.

I tried your code but I cannot reproduced in my environment which has selection custom field.

comment:3 Changed 18 years ago by anonymous

Thanks for the suggestion. Note the traceback is slightly different because I changed the exception to tell me which field had the problem but its the same exception. From trac.log on the server I get:

2006-11-21 07:09:28,115 Trac[web_ui] ERROR: Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/tracrpc/web_ui.py", line 58, in process_reque
st
  File "/usr/lib/python2.3/xmlrpclib.py", line 969, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.3/xmlrpclib.py", line 604, in dumps
    dump(v, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
    f(self, value, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 676, in dump_array
    dump(v, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 616, in __dump
    f(self, value, write)
  File "/usr/lib/python2.3/xmlrpclib.py", line 699, in dump_struct
    raise NameError,  k
NameError: verified

comment:4 Changed 18 years ago by Shun-ichi Goto

Yet I cannot reproduce your error although I've added same custom field 'verified' on trac 0.10.2 (on windows and python 2.3.5). I prefer to check (by logging, etc.) type of object in variable 'k' to ensure it is string or not when trapped before raise NameError (by your debug code, I guess) becaues the error "dictionary key must be string" is raised when 'k' is nether StringType nor UnicodeType (in xmlrpclib.dump_struct()).

comment:5 Changed 18 years ago by anonymous

Thank you for looking at this. I have verified the type and it is, in fact, unicode but xmlrpclib only checks for string, if it check for unicode it would be fine. See below (with my debug code in it):

    def dump_struct(self, value, write, escape=escape):
        i = id(value)
        if self.memo.has_key(i):
            raise TypeError, "cannot marshal recursive dictionaries"
        self.memo[i] = None
        dump = self.__dump
        write("<value><struct>\n")
        for k in value.keys():
            write("<member>\n")
            try:
              raise NameError,  type(k)
            except NameError:
              print k
            if type(k) is not StringType:
              try:
                  raise TypeError, "dictionary key must be string"
              except TypeError:
                raise NameError,  k
                raise
            write("<name>%s</name>\n" % escape(k))
            dump(value[k], write)
            write("</member>\n")
        write("</struct></value>\n")
        del self.memo[i]
    dispatch[DictType] = dump_struct

This is python 2.3.4-14.3 on CentOS 4U4

comment:6 Changed 18 years ago by anonymous

Just for fun, I added a check in for UnicodeType in xmlrpclib.py and now it seems to work. So I suppose the bug is really in xmlrpclib but any idea why custom fields are unicode but regular fields are string?

comment:7 in reply to:  5 Changed 18 years ago by Shun-ichi Goto

Replying to anonymous:

This is python 2.3.4-14.3 on CentOS 4U4

dump_struct() of python 2.3.5 is:

    def dump_struct(self, value, write, escape=escape):
        i = id(value)
        if self.memo.has_key(i):
            raise TypeError, "cannot marshal recursive dictionaries"
        self.memo[i] = None
        dump = self.__dump
        write("<value><struct>\n")
        for k, v in value.items():
            write("<member>\n")
            if type(k) is not StringType:
                if unicode and type(k) is UnicodeType:
                    k = k.encode(self.encoding)
                else:
                    raise TypeError, "dictionary key must be string"
            write("<name>%s</name>\n" % escape(k))
            dump(v, write)
            write("</member>\n")
        write("</struct></value>\n")
        del self.memo[i]

It seems there is a difference in unicode consideration between 2.3.4 and 2.3.5. Thus, easiest workaround is modifying xmlrpclib.py like above or upgrade to 2.3.5 if possible. It seems a bug of xmlrpclib.

The reason of name of non-custom fields are not unicode is field name literals in TicketSystem.get_ticket_fields() are not unicode string. On the other hand, strings read from trac.ini are all unicode. It is the difference, I guess.

Following patch might be help if we make workaround in xmlrpcplugin to support python 2.3.4 or erlier.

  • ticket.py

     
    6767    def get(self, req, id):
    6868        """ Fetch a ticket. Returns [id, time_created, time_changed, attributes]. """
    6969        t = model.Ticket(self.env, id)
    70         return (t.id, t.time_created, t.time_changed, t.values)
     70        values = {}
     71        for k, v in t.values.items():
     72            import sys
     73            if sys.version_info[:3] < (2,3,5) and isinstance(k, unicode):
     74                k = k.encode('utf-8') # workaround for old xmlrpclib
     75            values[k] = v
     76        return (t.id, t.time_created, t.time_changed, values)
    7177
    7278    def create(self, req, summary, description, attributes = {}):
    7379        """ Create a new ticket, returning the ticket ID. """

comment:8 Changed 18 years ago by greg@…

Indeed it seems to be a problem in the python distribution however since I guess alot of people will want to stay with their distribution's currently "blessed" version its probably better to fix the plugin code. Your patch above is great but that is only for the get method. Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

Thanks again for looking at this.

comment:9 in reply to:  8 Changed 18 years ago by Shun-ichi Goto

Replying to greg@max-t.com:

Do you know if there are other methods that will have the same issue? (eg: trying to set a custom field).

I cannot test to find good or not because I don't have 2.3.4 environment (windows and debian). Please test for other methods on your environment then report here.

comment:10 Changed 18 years ago by rob.sharp@…

I have patched the plugin as mentioned above, looks like it works well!

comment:11 Changed 15 years ago by osimons

#1718 closed as duplicate.

comment:12 Changed 14 years ago by Thijs Triemstra

Cc: Thijs Triemstra added; anonymous removed
Owner: changed from Alec Thomas to osimons

python 2.3 is very old and proably shouldn't be supported anymore?

comment:13 Changed 12 years ago by osimons

Resolution: worksforme
Status: newclosed

Problem is way too old... Whatever the problem was it is working now with newer versions. Just upgrade everything.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain osimons.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.