Modify

Opened 11 years ago

Closed 11 years ago

#11151 closed defect (fixed)

OSError: [Errno 13] Permission denied: '/srv/trac-hacks.org/trac/files'

Reported by: Ryan J Ollos Owned by: Michael Renzmann
Priority: normal Component: TracHacks
Severity: normal Keywords:
Cc: Jun Omae, Steffen Hoffmann, osimons Trac Release:

Description

There have been a few mailing list posts, including this recent one, about the Trac attachment migration not working. At least two of them were upgrading from 0.11 to 1.0. The migration doesn't seemed to have worked on t-h.o.

  1. When I try to add an attachment to a ticket, I get the traceback below. Not surprising since there is no files directory in the environment.
  2. There was no error during the trac-admin $ENV upgrade.
  3. The Image macro shows attachments, but it is not possible to navigate directly to an attachment (try attachment:wiki:AccountManagerPlugin:components-admin.png, which does exist).

We'll need some way to fix up the database now I assume. It would be nice to figure out why this isn't working in some cases and submit a patch to Trac.

drwxr-xr-x 12 root     root     4096 Jun  6 02:46 .
drwxr-xr-x  7 root     root     4096 Sep  6  2012 ..
drwxr-xr-x 16 www-data www-data 4096 Sep  5  2012 .egg-cache
drwxr-xr-x  2 www-data www-data 4096 Mar 22 17:03 .poll
drwxr-xr-x  2 www-data www-data 4096 Sep  4  2012 .reposearch
-rw-r--r--  1 root     root       98 Jan 23  2009 README
-rw-r--r--  1 root     root       27 Jan 23  2009 VERSION
drwxr-xr-x  4 www-data www-data 4096 Feb  4  2009 attachments
drwxr-xr-x  3 www-data www-data 4096 Jun  6 09:37 conf
drwxr-xr-x  2 root     root     4096 Jun  6 05:00 db
drwxr-xr-x  2 www-data www-data 4096 Jun  6 10:06 log
drwxr-xr-x  2 root     root     4096 Jun  6 02:46 plugins
drwxr-xr-x  2 root     root     4096 Mar  2  2009 templates
drwxr-xr-x  2 root     root     4096 Sep  6  2012 wiki-macros
File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/web/main.py", line 497, in _dispatch_request
  dispatcher.dispatch(req)
File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/web/main.py", line 214, in dispatch
  resp = chosen_handler.process_request(req)
File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/attachment.py", line 512, in process_request
  data = self._do_save(req, attachment)
File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/attachment.py", line 769, in _do_save
  attachment.insert(filename, upload.file, size)
File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/attachment.py", line 319, in insert
  os.makedirs(dir)
File "/usr/lib/python2.6/os.py", line 150, in makedirs
  makedirs(head, mode)
File "/usr/lib/python2.6/os.py", line 150, in makedirs
  makedirs(head, mode)
File "/usr/lib/python2.6/os.py", line 150, in makedirs
  makedirs(head, mode)
File "/usr/lib/python2.6/os.py", line 150, in makedirs
  makedirs(head, mode)
File "/usr/lib/python2.6/os.py", line 157, in makedirs
  mkdir(name, mode)

Attachments (0)

Change History (6)

comment:1 Changed 11 years ago by Ryan J Ollos

Cc: osimons added

comment:2 Changed 11 years ago by Jun Omae

Cannot reproduce....

The migration is defined in trac/upgrades/db28.py. First, the function checks the existence of $ENV/attachments. If it exists, run do_upgrade().

But, if the os.path.exists(old_dir) returns False by some reasons, e.g. selinux? nfs cache?, it does nothing. The os.path.exists() returns False if os.stat() raises any os.error, even if not ENOENT.

  • trac/upgrades/db28.py

    diff --git a/trac/upgrades/db28.py b/trac/upgrades/db28.py
    index 4079285..15f4091 100644
    a b  
    1111# individuals. For the exact contribution history, see the revision
    1212# history and logs, available at http://trac.edgewall.org/.
    1313
     14import errno
    1415import os.path
    1516
    1617from trac.attachment import Attachment
    def do_upgrade(env, version, cursor): 
    2324    the filenames in the process."""
    2425    path = env.path
    2526    old_dir = os.path.join(path, 'attachments')
    26     if not os.path.exists(old_dir):
    27         return
    28     old_stat = os.stat(old_dir)
     27    try:
     28        old_stat = os.stat(old_dir)
     29    except OSError, e:
     30        if e.errno == errno.ENOENT:
     31            return
     32        raise
    2933    new_dir = os.path.join(path, 'files', 'attachments')
    30     if not os.path.exists(new_dir):
     34    if not os.path.isdir(new_dir):
    3135        os.makedirs(new_dir)
    3236
    3337    cursor.execute("""

run-db28.py script

# -*- coding: utf-8 -*-
#
# Execute `do_upgrade` in trac/upgrades/db28.py
#
# Usage: python run-db28.py /path/to/tracenv
#

from __future__ import with_statement

from trac.env import Environment
from trac.upgrades import db28

def main(args):
    for arg in args:
        env = Environment(arg)
        with env.db_transaction as db:
            cursor = db.cursor()
            db28.do_upgrade(env, 28, cursor)

if __name__ == '__main__':
    import sys
    main(sys.argv[1:])
Last edited 11 years ago by Jun Omae (previous) (diff)

comment:3 Changed 11 years ago by Ryan J Ollos

Hi Jun, Thank you for the patch. I tried running your run-db28.py locally on separate instances of the trac-hacks environment that I pulled off the server. I tried on one instance with the db28.py patched, and one without. In both cases the attachment upgrade was successful. Finally, I patched the db28.py on the trac-hacks server and ran the upgrade and it was successful as well.

Here is the output, which looks pretty standard to me.

rjollos@hacks:/srv/trac-1.0.1$ sudo /srv/trac-hacks.org/pve/bin/python run-db28.py /srv/trac-hacks.org/trac
The upgrade of attachments was successful, but the old attachments directory:

  /srv/trac-hacks.org/trac/attachments

couldn't be removed, possibly due to the presence of files that weren't
referenced in the database. The error was:

  OSError: [Errno 39] Directory not empty: '/srv/trac-hacks.org/trac/attachments/wiki/MyProjectPlugin/Project Files'

This error can be ignored, but for keeping your environment clean you should
backup any remaining files in that directory and remove it manually.

To summarize, as others on the mailing list have reported, running db28.py after a failed upgrade seems to work well, however I can't say if the patch had an effect. I will see if I can pull down a copy of the Trac 0.10.6 environment, install that locally and reproduce the failed upgrade.

On the trac-hacks server we are running,

$ /etc/apache2/sites-common$ cat /etc/debian_version
6.0.7
$ uname -a
Linux hacks 2.6.22-3-vserver-686 #1 SMP Tue Dec 4 08:57:30 UTC 2007 i686 GNU/Linux

It looks like we are not running SELinux.

Last edited 11 years ago by Ryan J Ollos (previous) (diff)

comment:4 in reply to:  3 ; Changed 11 years ago by Jun Omae

Thanks for the details.

...., however I can't say if the patch had an effect.

Indeed, it seems that the patch has no effects.

I will see if I can pull down a copy of the Trac 0.10.6 environment, install that locally and reproduce the failed upgrade.

Ok. I hope to reproduce this issue.

BTW, is there trac.log of the Trac environment on trac-hacks.org which was upgraded on Jun 6? Could you please post the trac.log?

comment:5 in reply to:  4 Changed 11 years ago by Ryan J Ollos

Replying to jun66j5:

I will see if I can pull down a copy of the Trac 0.10.6 environment, install that locally and reproduce the failed upgrade.

Ok. I hope to reproduce this issue.

I can try to make a copy of the Trac 0.10.6 environment available to you if that would help.

BTW, is there trac.log of the Trac environment on trac-hacks.org which was upgraded on Jun 6? Could you please post the trac.log?

From what I see in /srv/trac-hacks.org/trac/log, there is a log rotate in place that retains 5 log files and we seemed to have just missed the opportunity to grab the one from June 6th. I'm not sure if logs are backed-up somewhere else for longer-term, but I can check into that.

comment:6 Changed 11 years ago by Ryan J Ollos

Resolution: fixed
Status: newclosed

trac:#11370 has been opened for this issue. Since the issue is resolved on t-h.o, I'll close the ticket for now.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Michael Renzmann.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.