[patch] Error after upgrade from 0.11 to trunk version

[ogiesen]

I just tested the upgrade from trac 0.11 to 0.12 on a local clone of our trac 0.11 installation. I upgraded using easy_install and everything went smooth (except the wiki editor is now mostly broken). At first everything appeared to be working fine with account manager as well. Then I saw that there was a dedicated version of account manager for trac 0.12 (previously we were using the version from the 0.11 branch) and ran easy_install https://trac-hacks.org/svn/accountmanagerplugin/trunk as specified on the plugin's home page. After that, when trying to access our trac environment all I get is the following stack trace:

Traceback (most recent call last):
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\web\api.py", line 436, in send_error
    data, 'text/html')
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\web\chrome.py", line 808, in render_template
    template = self.load_template(filename, method=method)
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\web\chrome.py", line 769, in load_template
    self.get_all_templates_dirs(), auto_reload=self.auto_reload,
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\web\chrome.py", line 481, in get_all_templates_dirs
    for provider in self.template_providers:
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\core.py", line 78, in extensions
    return filter(None, [component.compmgr[cls] for cls in extensions])
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\core.py", line 213, in __getitem__
    component = cls(self)
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\core.py", line 119, in maybe_init
    init(self)
  File "build\bdist.win32\egg\acct_mgr\web_ui.py", line 189, in __init__
    self._write_check(log=True)
  File "build\bdist.win32\egg\acct_mgr\web_ui.py", line 192, in _write_check
    writable = AccountManager(self.env).get_all_supporting_stores('set_password')
  File "c:\users\me\appdata\local\temp\easy_install-epr54e\Trac-0.12-py2.6-win32.egg.tmp\trac\core.py", line 119, in maybe_init
    init(self)
  File "build\bdist.win32\egg\acct_mgr\api.py", line 147, in __init__
    locale_dir = resource_filename(__name__, 'locale')
  File "C:\Program Files\Python26\lib\site-packages\pkg_resources.py", line 882, in resource_filename
    self, resource_name
  File "C:\Program Files\Python26\lib\site-packages\pkg_resources.py", line 1352, in get_resource_filename
    return self._extract_resource(manager, zip_path)
  File "C:\Program Files\Python26\lib\site-packages\pkg_resources.py", line 1363, in _extract_resource
    zip_stat = self.zipinfo[zip_path]
KeyError: 'acct_mgr\\locale'

As you can see, this is on Windows 7. We're using Apache with MOD_WSGI and the HtPasswdStore module. We also have the HttpAuthPlugin installed and enabled for the xmlrpc paths.

I've already tried uninstalling and reinstalling the account manager plugin but without success.

Any ideas?

[Steffen Hoffmann]

Replying to ogiesen:

![...] I've already tried uninstalling and reinstalling the account manager plugin but without success.

Would you dare to share some more details on how/what exactly you did/tried there, please?

Any ideas?

Several, actually. Just to name some for the start:

easy_install

This installation method hasn't been tested here by now, so your test especially appreciated. Anyway, consider downloading/check-out from SVN in a more old-fashioned way. If done so, run unit test on the source, please. I've taken care to keep them up-n-working, but tested on Debian GNU/Linux kernel 2.6.26 with Python2.5 only. And check for proper message catalog compilation. The KeyError: 'acct_mgr\\locale' suggests, that there has been a failure in generating any such compiled catalog file (*.mo) preventing the creation of the dedicated storage patch acct_mgr/locale altogether.

plugin interaction

While maybe not striking here, I just want to mention, there are 3(!) plugins to consider, i.e. see #1169, which seems related even to your use case with mod_wsgi and xmlrpc, but had received a wontfix by the HttpAuthPlugin maintainer.

Ticket rating

If mentioning trunk, provide the SVN rev. please.
This is subject to discussion: When applying trunk code here, expecting a flying start might have been a little bit too optimistic.
But as a matter of fact this is bleeding-edge and there's a lot of new/recently modified code. It's unreleased by now into any other branch, so I'll not allow 'blocker' status on it.

More we'll hopefully elaborate from your further information here, if this isn't enough, to pinpoint the issue within your install procedure or local configuration.

[John Hampton]

While upgrading one of my sites to 0.12-stable, I ran into this issue also. Haven't fixed it yet. I'm digging into it. I'll report back what I find.

[John Hampton]

OK, the problem is with zipped eggs. Installing the plugin globally, via python setup.py install and also using an unzipped-egg in the plugins dir of my project works.

I'll look a little more into the issue, as it's annoying to work with unzipped eggs as the need to be created manually (if not installing globally).

[John Hampton]

Ok, so, more digging. This is due to the lack of a locale directory in the zip file. Mainly due to the fact that I don't use any translations and so the line in setup.py​ doesn't match any files and the directory is then left out of the resulting egg file.

This is, apparently, a known issue as the AccountManagerPlugin wiki page says:

Complaints about missing locale directory are often a side-effect of failure
to compile any message catalog for inclusion into Python egg, hence the
whole path is missing. Due to a know Trac issue Babel has to be installed
prior to Trac, to get it all working as expected.

Babel shouldn't be required just to get a plugin to work as an egg. Especially if no translations are needed.

A simple solution is to add a .keep file in the locale directory and adjust the setup.py such that the locale directory is always included in the egg.

acct_mgr/locale/.keep

@@ +1 @@
+# Need someting in the .zip so it doesn't blow-up on load

setup.py

@@ -36 +36 @@
     zip_safe=True,
     packages=['acct_mgr'],
     package_data={'acct_mgr': ['locale/*/LC_MESSAGES/*.mo',
+                               'locale/.keep',
                                'templates/*.html',
                                'templates/*.txt']},
 

This issue also affects the TagsPlugin.

If there are no objections, I'll commit the patch.

[ogiesen]

Replying to hasienda:

If mentioning trunk, provide the SVN rev. please.
This is subject to discussion: When applying trunk code here, expecting a flying start might have been a little bit too optimistic.
But as a matter of fact this is bleeding-edge and there's a lot of new/recently modified code. It's unreleased by now into any other branch, so I'll not allow 'blocker' status on it.

Understood. To be honest I simply followed the instructions in the "For trac 0.12" section of AccountManagerPlugin#Install without really thinking much about the trunk implications. Sorry about that. Seems quite stupid in hindsight...
Please keep that in mind when reading the rest of my comments below.

Would you dare to share some more details on how/what exactly you did/tried there, please?

Well, I was just following the instructions on ​t:TracPlugins#Uninstall . I'm not a plugin hacker myself.

easy_install

This installation method hasn't been tested here by now, so your test especially appreciated.

Well, this is a bit surprising as it's the only method advertised on AccountManagerPlugin#Install

Anyway, consider downloading/check-out from SVN in a more old-fashioned way. If done so, run unit test on the source, please. I've taken care to keep them up-n-working, but tested on Debian GNU/Linux kernel 2.6.26 with Python2.5 only. And check for proper message catalog compilation. The KeyError: 'acct_mgr\\locale' suggests, that there has been a failure in generating any such compiled catalog file (*.mo) preventing the creation of the dedicated storage patch acct_mgr/locale altogether.

Sorry, I'm quite far from being a Python hacker so most of what you wrote above unfortunately doesn't mean a whole lot to me at the moment... I'm just the guy maintaining our trac server. I only write the occasional script or macro in Python and do my application development in other languages.

While maybe not striking here, I just want to mention, there are 3(!) plugins to consider, i.e. see #1169, which seems related even to your use case with mod_wsgi and xmlrpc, but had received a wontfix by the HttpAuthPlugin maintainer.

Well, the HttpAuthPlugin was already working fine together with the AccountManagerPlugin and the htpasswd file as long as we were on the trac 0.11 branch of the AccountManagerPlugin, so I don't think that ticket applies here.

More we'll hopefully elaborate from your further information here, if this isn't enough, to pinpoint the issue within your install procedure or local configuration.

To clarify: There was no sign that the installation procedure was unsuccessful in any way. The stack trace I posted is what I get when I try to access the trac instance in the browser.

Furthermore, if using the trac 0.11-branch of the plugin with trac 0.12 is the recommended procedure for the time being then that's perfectly fine with me. I only switched to the trunk version because that was what AccountManagerPlugin#Install seemed to recommend for trac 0.12.

[ogiesen]

Replying to pacopablo:

A simple solution is to add a .keep file in the locale directory and adjust the setup.py such that the locale directory is always included in the egg.

I can confirm that this fixed the reported issue for me. Thanks!

[stevek]

Replying to ogiesen:

Replying to pacopablo:

A simple solution is to add a .keep file in the locale directory and adjust the setup.py such that the locale directory is always included in the egg.

I can confirm that this fixed the reported issue for me. Thanks!

This only partially worked for me. I still get the KeyError: acct_mgr/locale traceback, but it's intermittent. In fact, I can usually make it appear and disappear by clicking the reload button a couple of times. This is not a cache issue, since I've reproduced this on other machines/browsers.

[John Hampton]

Replying to stevek:

Replying to ogiesen:

Replying to pacopablo:

A simple solution is to add a .keep file in the locale directory and adjust the setup.py such that the locale directory is always included in the egg.

I can confirm that this fixed the reported issue for me. Thanks!

This only partially worked for me. I still get the KeyError: acct_mgr/locale traceback, but it's intermittent. In fact, I can usually make it appear and disappear by clicking the reload button a couple of times. This is not a cache issue, since I've reproduced this on other machines/browsers.

This screams of a cache or left-over process issue. How is trac configured? Are you using mod_python or mod_wsgi with multiple processes?

Try restarting you webserver and making sure that all the trac processes are killed and any old code removed and then restart the webserver.

Additionally, if you are using mod_python or mod_wsgi you can try testing with a single tracd process and see if you can reproduce the error.

[Steffen Hoffmann]

Replying to pacopablo:

Ok, so, more digging. This is due to the lack of a locale directory in the zip file. ![...] This is, apparently, a known issue ![...]

Correct, but I did remember only after this ticket has been opened and added the cited paragraph to the wiki page while commenting here.

Babel shouldn't be required just to get a plugin to work as an egg. Especially if no translations are needed.

Agree, even if it sounds not equally nice to all developers. I know this is pragmatism, but could be easily mistaken as arrogance by others that feel English dominance as pressure against their own beloved native language, you understand?

After all i18n is an offer to the Trac user, and for many applications it's not exactly clear, that they're English-only until the end of time. Even en_US vs. en_GB may become an interesting option for developers out there. Anyway, I just want to soften stances that could be read as 'English/Trac without Babel still remains the major use case to take care for'. I'll try hard to handle every reasonable use case with equal respect.

A simple solution is to add a .keep file in the locale directory ![...]

Ok, I'm still wondering, why I've never seen this issue with Trac itself. But since I couldn't figure this out by quick research this evening (setuptools documentation seems to be 'for packaging gurus only'), we'll adopt this solution.

This issue also affects the TagsPlugin.

Thanks for the hint. Will take care of this as well.

If there are no objections, I'll commit the patch.

Not at all, but since I've already prepared other changes for setup.py hopefully you don't mind me committing this myself with reference to your patch proposal here.

Thank you very much for testing and debugging effort.

[anonymous]

Replying to pacopablo:

Replying to stevek:

Replying to ogiesen:

Replying to pacopablo:

A simple solution is to add a .keep file in the locale directory and adjust the setup.py such that the locale directory is always included in the egg.

I can confirm that this fixed the reported issue for me. Thanks!

This only partially worked for me. I still get the KeyError: acct_mgr/locale traceback, but it's intermittent. In fact, I can usually make it appear and disappear by clicking the reload button a couple of times. This is not a cache issue, since I've reproduced this on other machines/browsers.

This screams of a cache or left-over process issue. How is trac configured? Are you using mod_python or mod_wsgi with multiple processes?

Try restarting you webserver and making sure that all the trac processes are killed and any old code removed and then restart the webserver.

Additionally, if you are using mod_python or mod_wsgi you can try testing with a single tracd process and see if you can reproduce the error.

I reinstalled and restarted Apache. It no longer produces this error. Thanks!

[John Hampton]

Replying to hasienda:

Replying to pacopablo:

Babel shouldn't be required just to get a plugin to work as an egg. Especially if no translations are needed.

Agree, even if it sounds not equally nice to all developers. I know this is pragmatism, but could be easily mistaken as arrogance by others that feel English dominance as pressure against their own beloved native language, you understand?

After all i18n is an offer to the Trac user, and for many applications it's not exactly clear, that they're English-only until the end of time. Even en_US vs. en_GB may become an interesting option for developers out there. Anyway, I just want to soften stances that could be read as 'English/Trac without Babel still remains the major use case to take care for'. I'll try hard to handle every reasonable use case with equal respect.

The default "locale" on any no-localized machine, is POSIX, or C, which happens to be English. However, that's not really the issue. Babel is a developers tool, not a user tool. Babel shouldn't be required by any user, whether or not they are using localization. Of course, that gets into a larger discussion about distribution, etc. which I don't really think is necessary.

A simple solution is to add a .keep file in the locale directory ![...]

Ok, I'm still wondering, why I've never seen this issue with Trac itself. But since I couldn't figure this out by quick research this evening (setuptools documentation seems to be 'for packaging gurus only'), we'll adopt this solution.

By default, when one installs Trac proper, it doesn't get installed as a zipped egg. This problem only rears its head if the egg is zipped. I haven't tried forcing Trac to be a zipped egg, but I suspect that it may be susceptible to this issue too.

Not at all, but since I've already prepared other changes for setup.py hopefully you don't mind me committing this myself with reference to your patch proposal here.

Have at it.

[Steffen Hoffmann]

(In [9330]) AccountManagerPlugin: Make sure to always include a 'locale' dir in zipped plugins, closes #7850.

Thanks to pacopablo for providing an instant patch. I've taken the chance to rework it a little and do some more changes to setup.py.

[Steffen Hoffmann]

(In [9331]) AccountManagerPlugin: Follow-up, adding the file I missed in changeset [9330], refs #7850.

[Steffen Hoffmann]

Replying to hasienda:

Replying to pacopablo: ![...]

This issue also affects the TagsPlugin.

Thanks for the hint. Will take care of this as well.

Done in changeset [9332].