HttpAuthPlugin not redirecting authentication request to httpd

[Joe Kiniry]

We have setup several Trac servers here in the ​School of Computer Science and Informatics at ​University College Dublin, one of which is support software development for ​a major EU FP6 grant. Thanks very much for this excellent software.

That being said, we have one small problem. Apologies for the priority and severity, but these issues are blocking some major work and server migrations happening at the moment here at UCD.

We have successfully configured about 25 different and very useful plugins and macros. The Trac servers run on an Apple XServer running OS X Server (Darwin 8.10.1 and OS X Server 10.4.10). We are using the Trac 0.10 install on Python 2.5 with Apache 2.2 and SQLite 3, all of which were installed via MacPorts 1.5. We prefer to use stable releases. All Trac access to httpd is via SSL. Insecure access via the HTTP protocol are redirected to HTTPS.

The core plugins involved with our problem with ​HttpAuthPlugin are (we suspect) ​PermRedirect, ​AccountManager, and ​XmlRpc. We are attempting to permit access to the problematic Trac via AccountManager's LoginModule, we only wish to permit authenticated access (and thus, the use of PermRedirect), and authentications are meant to happen via HttpAuth to an authentication URL on the httpd server that performs LDAP-based httpd authentication. We also need full XmlRpc access for our heavy Mylyn use.

Authenticating manually against the authentication URL /auth works correctly, thus LDAP authentication in httpd is fine. Attempting to access any part of the site without authentication properly redirects to the /login URL. Attempting to authenticate to the HTML-based login form provided by LoginModule fails.

Examining the server log during authentication shows that, while HttpAuth is loaded and activated, its attempt at authentication fails in HTTPAuthFilter, and moreover, the HTTPAuthFilter never attempted to perform an HTTP GET on the /auth URL at all.

I have even gone so far as to annotate AccountManager and HttpAuth plugin code in an attempt to trace the problem. I have also compared the 0.10 releases of the relevant plugins to their trunks to see if any bugfixes in the 0.11 dev tree are relevant.

I am honestly a bit out of my comfort-zone, given I work mainly in formal verification of over-documented, formally specified, compiled, heavily typed languages. So, while quite cool and useful, Trac does not resemble those kinds of systems very much. :)

Anyone have any ideas?

Relevant parts of trac.ini can be provided if necessary, but performing a debug log shows all the proper plugins are loading. We are seeing some warning and error messages in the Trac logfile, but we believe that they are not relevant. They are appended below, just in case.

Thanks, ​Joe Kiniry

--- trac.log snippit:

AttributeError: 'NoneType' object has no attribute 'get'
2007-09-12 00:43:57,912 Trac[main] ERROR: 'BatchModifyModule' object has no attribute 'get_htdocs_dirs'
Traceback (most recent call last):
  File "/opt/local/lib/python2.5/site-packages/trac/web/main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "/opt/local/lib/python2.5/site-packages/trac/web/main.py", line 237, in dispatch
    resp = chosen_handler.process_request(req)
  File "/opt/local/lib/python2.5/site-packages/trac/web/chrome.py", line 226, in process_request
    in provider.get_htdocs_dirs() if dir[0] == prefix]:
AttributeError: 'BatchModifyModule' object has no attribute 'get_htdocs_dirs'

[Noah Kantrowitz]

You have two problems. One, you are using the BatchModifyPlugin which is broken and can't be used. Two, the HttpAuthPlugin doesn't do what you think it does. You want to use the ~HttpAuth backend for AccountManager. The HttpAuthPlugin is what allows you to use things like the XmlRpcPlugin, which need HTTP authentication, with AccountManager's form-based logins.

[Joe Kiniry]

Thank you for the quick and efficient response. I am sorry, I was a bit imprecise, and it was late at night when I filed this ticket. I confused our simultaneous use of the HttpAuthStore AccountManagerPlugin backend with HttpAuthPlugin.

First, thank you for the information on BatchModifyPlugin. I have removed it from our configuration.

Second, we are indeed attempting to use the HttpAuthStore backend for AccountManagerPlugin to authenticate form-based login in the manner in which I described. We are also using the HttpAuthPlugin to authenticate XmlRpcPlugin access via this (dual-layer, in a sense) backend. This clarification explains why I am not seeing an HTTP GET request when testing only the form login---I should see such only when (perhaps manually) authenticating via XML-RPC. I will take that into account in further testing today (see below).

Here are the relevant sections of trac.ini:

[account-manager]
authentication_url = https://srg-trac.ucd.ie/auth
password_store = HttpAuthStore

[components]
acct_mgr.api.accountmanager = enabled
acct_mgr.http.httpauthstore = enabled
acct_mgr.web_ui.loginmodule = enabled
permredirect.* = enabled
permredirect.filter.permredirectmodule = enabled
trac.web.auth.loginmodule = disabled
tracauthrequired.authrequired.authrequired = enabled
tracrpc.api.xmlrpcsystem = enabled
tracrpc.search.searchrpc = enabled
tracrpc.ticket.componentrpc = enabled
tracrpc.ticket.milestonerpc = enabled
tracrpc.ticket.priorityrpc = enabled
tracrpc.ticket.resolutionrpc = enabled
tracrpc.ticket.severityrpc = enabled
tracrpc.ticket.statusrpc = enabled
tracrpc.ticket.ticketrpc = enabled
tracrpc.ticket.typerpc = enabled
tracrpc.ticket.versionrpc = enabled
tracrpc.web_ui.xmlrpcweb = enabled
tracrpc.wiki.wikirpc = enabled
tracrpcext.ticket.ticketextrpc = enabled
tracrpcext.wiki.wikiextrpc = enabled

[httpauth]
paths = /xmlrpc, /login/xmlrpc

(As an aside, I'm not entirely clear on the semantics of the Trac configuration file. Does foo.* = enabled enabled all settings, recursively, under foo, or something else? I.e., if I wish to enable all RPC-related functionality can one do tracrpc.* = enabled and/or tracrpc.*.* = enabled or similar, or am I simply reading too much into the semantics of dot?)

Additionally, Apache's configuration has a <Location> section that performs the LDAP-based authentication:

<Location /auth>
    AuthName "UCD Trac"
    AuthType Basic
    AuthBasicProvider ldap
    AuthLDAPURL ldap://localhost/dc=kind,dc=ucd,dc=ie?uid
    AuthzLDAPAuthoritative off
    # require ldap-group cn=perverts,dc=ucd,dc=ie
    # setting valid-user means everyone with a valid LDAP login can
    # access this trac.  if set then AuthzLDAPAuthoritative must be
    # off.
    require valid-user
  </Location>

You'll note that we still have not been successful in authenticating against a specific GID on the server, but I'm unconcerned about that for now.

After a morning meeting I hope to re-check the server again with a clear head and with your clarifying statement in-mind. I am going to reopen this ticket until such time as I solve this configuration issue and I will document our resolution here as well as add clarifying notes, where appropriate, to the relevant plugin wiki pages.

[Noah Kantrowitz]

Well for starters you don't have the HttpAuthPlugin enabled at all.

[Joe Kiniry]

I missed a relevant cut-and-paste from the configuration (as the whole of it is much larger than what I have shown, of course). It is enabled:

httpauth.* = enabled

I have found the time to work on this again today. More updates soon. Thanks.

[Joe Kiniry]

While I have solved (or, at least, cleaned up) some related problems, authentication with the HttpAuthStore backend to AccountManagerPlugin still does not work.

First, I have moved the /auth URL to another VirtualHost so that the Trac's / Location does not override it. Authenticating to this new /auth URL via HTTPS works as expected.

Thus, my account-manager section now reads:

[account-manager]
authentication_url = https://kind.ucd.ie/auth/
password_store = HttpAuthStore

I hope the use of HTTPS here is not problematic. This may be an issue; I'm reading the Python docs on urllib2 to attempt to understand if this is a problem.

Second, to avoid confounding issues, I have wholly disabled the XmlRpcPlugin and the PermRedirectPlugin.

Now, the only warning the Trac (DEBUG-level) log is:

Trac[web_ui] WARNING: AccountModule is disabled because the password store does not support writing.

Looking at the source for AccountManagerPlugin, this seems to indicate only that the chosen backend, in this case HttpAuthStore, does not support password updates. The implications of this fact (one cannot change or reset passwords or delete accounts) is fine for our situation.

Unfortunately, all that being the case, the HttpAuthStore backend of AccountManagerPlugin still does not authenticate for me.

In fact, this debug log message in HttpAuthStore is never even printed to the Trac logfile:

        self.log.debug('HttpAuthStore: Attempting to authenticate user %s against \
auth_url %s', user, self.auth_url, user)

(It is not clear to me how HttpAuthStore gets activated an installed in the first place, given a grep-find across the whole of AccountManagerPlugin's code has no matches for the classname HttpAuthStore. I presume it happens in the AccountManager class's initialization in api.py by virtue of the property password_store's value.)

AccountManagerPlugin's http extension is definitely being loaded:

Trac[loader] DEBUG: Loading egg plugin acct_mgr.http from /opt/local/lib/python2.5/site-packages/TracAccountManager-0.1.3dev_r2548-py2.5.egg

Still thinking/working on this...

[Joe Kiniry]

The HttpAuthStore plugin is simply never being loaded/used. Debug messages put into its check_password are never printed to the Trac log.

How does one debug such issues? Is this at all related to the warning about AccountModule being disabled?

There seems to be some non-deterministic behavior happening here...

/me still debugging.

[Joe Kiniry]

After a debugging on and off today, we have solved the problem.

The issue was, at its core, threefold:

1. The ​MacPorts Python 2.5 build does not include SSL, nor does it have a configuration options for such. Thus, attempting to use an HTTPS URL for auth_url silently fails.
2. After figuring out the above, I tried to ensure that access to the /auth URL was restricted to only processes running on localhost. My Apache Allow from directive stated 127.0.0.1, not realizing that localhost loopback connects still have actual interface information embedded in them for deny-checking. Thus, the proper line was: Allow from 127.0.0.1 137.43.92.49, where the latter IP is the IP of the server itself. The finally,
3. The reason we believed that the plugin was not loading at all was because the self.log.debug message was never seen in the log. This was caused by a third problem (endemic to these kinds of languages, IMO): that line has a print descriptor expecting two strings, but the varargs had three parameters, and Python causes such a statement to silently fail. <sigh>

I am now closing this ticket and hope that this log will help others that might have similar challenges getting this synthesis of plugins working.

Thanks again.

Joe