wiki:XmlRpcPlugin

Version 87 (modified by kritter@…, 3 years ago) (diff)

Added remark that xmlrpclib has been renamed in Python 3

Trac XML-RPC Plugin

Remote Procedure Call plugin for Trac 0.10 (not actively maintained) and 0.11/0.12 (trunk).

Description

This plugin allows Trac plugins to export select parts of their interface via XML-RPC and JSON-RPC (if json or simplejson is available). Latest trunk version includes a pluggable API for extending protocols, and see for instance TracRpcProtocolsPlugin for more protocols.

The browsable XML-RPC URI suffix is /rpc (and /xmlrpc for older versions), however most XML-RPC clients should use the authenticated URL suffix /login/rpc as this is correctly authenticated by Trac.

The XML_RPC permission is used to grant users access to using the RPC interface. If you do want to use /rpc and unauthenticated access, you must grant the XML_RPC permission to the 'anonymous' user.

Method status:

  • Ticket API is also complete, with the following types exported: component, version, milestone, type, status, resolution, priority and severity.
  • WikiRPC API is complete, mostly thanks to mgood.

For example, for TracHacks the URIs are http://trac-hacks.org/xmlrpc and http://trac-hacks.org/login/xmlrpc (must be authenticated). (XML-RPC is currently disabled at trac-hacks.org)

Todo

Outstanding tasks are roadmap, timeline, user management (e.g. get a (filtered) user list to assign a task in mylyn), plugin management (?)...plus probably more.

Installation

This plugin requires at least Trac 0.10, but Trac 0.11 or 0.12 is recommended. Install in the same manner as any other Trac plugin:

# python setup.py bdist_egg 
# cp dist/*.egg /srv/trac/env/plugins 

or if you want it to be installed for all Trac environments (same command can be run later to refresh installation):

$ easy_install -Z -U http://trac-hacks.org/svn/xmlrpcplugin/trunk # 0.11+0.12
$ #or
$ easy_install -Z -U http://trac-hacks.org/svn/xmlrpcplugin/0.10 # 0.10
$ # or
$ easy_install -Z -U /path/to/unpacked/download/version 

You will also need to enable the plugin in your environments trac.ini:

[components] 
tracrpc.* = enabled 

Bugs/Feature Requests

Existing bugs and feature requests for XmlRpcPlugin are here. If you have any issues, create a new ticket.

Troubleshooting

Problems when AccountManagerPlugin is enabled

If you have the AccountManagerPlugin enabled and you followed their advice/example to disable the standard login module with

[components] 
trac.web.auth.LoginModule = disabled 

the /login/xmlrpc URL for authorized access will not work as expected. Every access will look like anonymous access. You can use the HttpAuthPlugin to correct this.

Problems with Digest HTTP authentication

The xmlrpclib.ServerProxy client - as demonstrated in the following examples - will not work with a Digest-based HTTP authentication: you need to set up a Basic HTTP authentication on server side to make the examples work.

If you use the standalone Trac daemon, this means that you cannot use the tracd -a option (htdigest authentication file). Use trac --basic-auth (htpasswd authentication file) instead.

Problems with mod_python, Apache, python 2.4

XmlRpcPlugin might not work with Apache and python 2.4 as explained in TracInstall. Use python 2.5 if you want to run Trac with mod_python.

Download and Source

Download the zipped source, check out the source using Subversion or browse the source with Trac.

Experimental features and work in progress can be found at a patches repository hosted by Bitbucket. Work in progress is developed using Mercurial Queues.

Example

NOTE: The xmlrpclib module has been renamed to xmlrpc.client in Python 3.0.

See http://docs.python.org/library/xmlrpclib.html for further information.

Python End-User Usage

Obtain and print a list of XML-RPC exported functions available to my user:

import xmlrpclib 
     
server = xmlrpclib.ServerProxy("http://athomas:password@localhost/trac-dev/login/xmlrpc") 
for method in server.system.listMethods(): 
  print method 
  print '\n'.join(['  ' + x for x in server.system.methodHelp(method).split('\n')]) 
  print 
  print 

The same example using system.multicall(). This reduces network and server load by compacting all of the system.methodHelp() calls into one HTTP POST.

import xmlrpclib 
     
server = xmlrpclib.ServerProxy("http://athomas:password@localhost/trac/devel/login/xmlrpc") 
         
multicall = xmlrpclib.MultiCall(server) 
for method in server.system.listMethods(): 
    multicall.system.methodHelp(method) 
     
for help in multicall(): 
    lines = help.splitlines() 
    print lines[0] 
    print '\n'.join(['  ' + x for x in lines[2:]]) 
    print 
     

List all tickets that are owned by athomas, using the XML-RPC multicall system to issue multiple RPC calls with one HTTP request:

import xmlrpclib 
         
server = xmlrpclib.ServerProxy("http://athomas:password@localhost/trac/devel/login/xmlrpc") 
         
multicall = xmlrpclib.MultiCall(server) 
for ticket in server.ticket.query("owner=athomas"): 
    multicall.ticket.get(ticket) 
print map(str, multicall()) 

Access the Wiki with WikiRPC

import xmlrpclib 
         
server = xmlrpclib.ServerProxy("http://athomas:password@localhost/trac-dev/login/xmlrpc") 
     
# print the content of WikiStart 
print server.wiki.getPage("WikiStart") 
     
# print WikiStart as HTML 
print server.wiki.getPageHTML("WikiStart") 
     
# write to the SandBox page from a text file 
sandbox_content = file("sandbox.txt").read() 
server.wiki.putPage("SandBox", sandbox_content, {"comment": "testing the WikiRPC interface"}) 

Add an attachment to WikiStart:

import xmlrpclib 
 
server = xmlrpclib.ServerProxy("http://athomas:password@localhost:8080/trunk/login/xmlrpc") 
     
server.wiki.putAttachment('WikiStart/t.py', xmlrpclib.Binary(open('t.py').read())) 

Using Digest Authentication in python

One can use digest authentication if you know the realm that you're connecting to. This shows up in the login box "server says '<realm'".

class HTTPSDigestTransport(xmlrpclib.SafeTransport):
    """
    Transport that uses urllib2 so that we can do Digest authentication.
    
    Based upon code at http://bytes.com/topic/python/answers/509382-solution-xml-rpc-over-proxy
    """

    def __init__(self, username, pw, realm, verbose = None, use_datetime=0):
        self.__username = username
        self.__pw = pw
        self.__realm = realm
        self.verbose = verbose
        self._use_datetime = use_datetime

    def request(self, host, handler, request_body, verbose):
        import urllib2

        url='https://'+host+handler
        if verbose or self.verbose:
            print "ProxyTransport URL: [%s]"%url

        request = urllib2.Request(url)
        request.add_data(request_body)
        # Note: 'Host' and 'Content-Length' are added automatically
        request.add_header("User-Agent", self.user_agent)
        request.add_header("Content-Type", "text/xml") # Important

        # setup digest authentication
        authhandler = urllib2.HTTPDigestAuthHandler()
        authhandler.add_password(self.__realm, url, self.__username, self.__pw)
        opener = urllib2.build_opener(authhandler)

        #proxy_handler=urllib2.ProxyHandler()
        #opener=urllib2.build_opener(proxy_handler)
        f=opener.open(request)
        return(self.parse_response(f))

digestTransport = HTTPSDigestTransport("username", "password", "realm")
server = xmlrpclib.ServerProxy("https://host/login/xmlrpc", transport=digestTransport)

Using from C#

See DotNet.

Using from Java

See this example zip file

Using from Ruby

You can either use the XMLRPC functionality included in the Ruby Standard Library or download the trac4r library which does all the trivial stuff for you.

please refer to the SSL Support section if you need one.

trac4r Example

This example uses trac4r:

require 'trac4r/trac'
# initialize the connection (username and password can be ommitted if not needed, but most of the time you will need them if anonymous doesn't have XMLRPC permissions)
trac = Trac.new "https://dev.example.com/trac/my_awesome_project", "myusername", "mypassword"
# get a list of all tickets (as an array of numbers)
trac.tickets.list :include_closed => true # this is the default anyway
# get all the tickets
# NOTE: the results here are cached, so you can call it as often as you want without producing traffic more than once.
# use ":cached_results => false" to get the latest version
trac.tickets.get_all :include_closed => true
# get a single ticket
ticket = trac.tickets.get 5
# print the data
puts "Title: #{ticket.summary}"
puts "Description: #{ticket.description}"
puts "Milestone: #{ticket.milestone}"
puts "Status: #{ticket.status}"
puts "Type: #{ticket.type}"
puts "Priority: #{ticket.priority}"
# get a list of all wiki pages
trac.wiki.list
# download one page
trac.wiki.get_html "SomeRandomPageName" # HTML version
trac.wiki.get_raw "AnotherRandomPageName" # trac syntax version (e.g. for editing)
# for previews use
trac.wiki.raw_to_html "content of a page in [wiki:WikiFormatting Trac syntax] as a ''String''"
# to post a page use
trac.wiki.put "NameOfThePage", "content in Trac syntax"
# list the attachments of a wiki page
trac.wiki.attachments "NameOfThePage"
# save an attachment
File.new("my_cool_document","w") do |f|
  f.write trac.wiki.get_attachment "NameOfThePage", "my_cool_document.pdf"
end

# upload an attachment to the page above
page = "NameOfThePage"
fn = "my_nice_doc.pdf"
fh = File.new(fn, "rb")
sdata = fh.read
data = XMLRPC::Base64.new(sdata)
result = trac.wiki.put_attachment(page, name, "uploaded via ruby script", data)
# the correct result should be the name of the file:
puts "ERROR: uploading #{name} didn't work properly!" if result != name

Also see the included Documentation in trac4r/doc

If you need to do a custom query do

trac.query("system.getAPIVersion")

The first argument is the method name, all other arguments are directly passed to XMLRPC. For this example of cause you could do trac.api_version instead ;)

If you have any problems with trac4r you can email me: niklas.cathor (eat) gmail dot com

xmlrpc Example

An example using XML-RPC directly with Ruby to append to a wiki page:

require 'xmlrpc/client'
user = "username"
password = "password"
page_name = "SandBoxRPC"
new_content = "\n\nMy new content"
new_comment = "Adding new content from ruby"

server = XMLRPC::Client.new2("https://#{user}:#{password}@trac.server.com/trac/login/xmlrpc")
content = server.call("wiki.getPage", page_name) + new_content
server.call("wiki.putPage", page_name, content, {"comment" => new_comment})

SSL Support + X509

The standard ruby xmlrpc client for some reason does not support SSL at all. In order to be able to use the XMLRPC over SSL with both

  • Client Certificate authentication
  • Basic Authentication

One must do as follows.

in order to proceed, you should have:

  • ca certificate in .pem format
  • personal certificate + RSA key in .p12 format (and its password, of course)
  • patched version of xmlrpc/client.rb
  • patched version of trac4r
  • username/password (for basic authentication)

Assuming you have the above, and your ca and user certificates are respectively:

~/.openssl/cacert.pem
~/.openssl/certkey.p12
The Instructions
  1. apply the patches:
    1. to the xmlrpc client
      sudo su -
      cd /usr/lib/ruby/1.8/
      patch -p0  < /path/to/xmlrpc-client.SSL.patch
      
    2. to the trac4r gem
      sudo su -
      cd /var/lib/gems/1.8/gems/
      patch -p0  < /path/to/trac4r-1.2.3.SSL.patch
      
  2. create a .yml file called ~/.trac/creds.yml of the following structure:
    --- 
    tracurl: https://yourserver.yourdomain/yourproject
    tracuser: yourwebuser
    tracpass: yourwebpassword
    certkey: /home/youruser/.openssl/certkey.p12
    cacert: /home/youruser/.openssl/cacert.pem
    keypass: yourkeypassword
    
    • do not forget to chmod the personal files to 600 or 400
  3. use the data in the code
    require 'yaml'
    require 'openssl'
    require 'xmlrpc/client'
    require 'trac4r'
    
    ## read the data from yaml:
    $ymlname= "#{ENV['HOME']}/.trac/creds.yml"
    if !File.exists?($ymlname) 
        raise "Cannot open credentials file!"
    end
    begin
        $vars = YAML::load_file($ymlname)
    rescue Exception => e
        raise "Cannot load credentials file #{$ymlname}: #{e.message}\nTrace: #{e.stacktrace}"
    end
    
    ## extract the certificate, and the key from the fles.
    pkcs = OpenSSL::PKCS12.new(File.open($vars['certkey']),$vars['keypass'])
    cert = pkcs.cert
    key = pkcs.key
    ## connect to the server
    trac = Trac.new($vars['tracurl'], $vars['tracuser'], $vars['tracpass'], $vars['cacert'], cert, key)
    ## from now you can refer to the connection as open (or query it)
    ## use the API as explained above.
    

NOTE: The working environment of the code (trac4r + ssl) is:

  • Debian/GNU system: Linux hostname 2.6.26-2-686 #1 SMP Mon Jun 21 05:58:44 UTC 2010 i686 GNU/Linux
  • OpenSSL: OpenSSL 0.9.8g 19 Oct 2007
  • Ruby: ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
  • The certificates have been issued by the above OpenSSL setup.

API Usage

See the source for details.

Screenshot

If the HTTP request to this URI is not XML, the XmlRpcPlugin will list all exported functions that the current user has permission to use.

Change Log

[13776] by osimons on 2014-03-19 00:06:29
XmlRpcPlugin: Tweak [13728] fix again to let carriage return pass through. Closes #11050 again. Also closes #11635.
[13729] by osimons on 2014-03-07 09:55:24
XmlRpcPlugin: Tweak [13728] fix by not removing surrogate pairs. Closes #11050 again.

Thanks Jun.

[13728] by osimons on 2014-03-07 00:49:04
XmlRpcPlugin: Remove invalid XML characters from ouput. Closes #11050.

Thanks olemis.

[13710] by osimons on 2014-02-21 09:57:47
XmlRpcPlugin: Added a test for ticket.type.getAll.
[13637] by osimons on 2014-02-02 16:55:06
XmlRpcPlugin: Fixed missing mimetype when returning error for unknown faults. Closes #11321.

Thanks to mattinflection for report + patch, and to AllenB for verification.

Author/Contributors

Authors and contributors: athomas, mgood, osimons, Olemis Lang
Maintainer: osimons

Attachments (2)

Download all attachments as: .zip