Modify

Opened 6 years ago

Closed 12 months ago

Last modified 12 months ago

#9610 closed defect (worksforme)

Upgrade errors from pastebin and/or downloads plugins

Reported by: Jonas Bardino Owned by: Ryan J Ollos
Priority: normal Component: DownloadsPlugin
Severity: major Keywords: upgrade failure
Cc: Jonas Bardino Trac Release: 0.11

Description

I'm not sure if it is a bug in the DownloadsPlugin, TracPastePlugin or core Trac, but I can consistently make trac-admin upgrade fail on the second invocation when enabling *both* TracPastePlugin and DownloadsPlugin at once for an existing Trac ENV.

All attempts to upgrade or access the environment on the web fails with the please upgrade message afterwards!

The upgrade output follows here:

0|~ > trac-admin /home/mig/state/vgrid_public_base/MiG-developers/.vgridtracker/var upgrade             mig@distlab4
/usr/lib/pymodules/python2.6/pyPdf/pdf.py:52: DeprecationWarning: the sets module is deprecated
  from sets import ImmutableSet
/usr/lib/pymodules/python2.6/graphviz/graphviz.py:23: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Upgrade done.
0|~ > trac-admin /home/mig/state/vgrid_public_base/MiG-developers/.vgridtracker/var upgrade             mig@distlab4
/usr/lib/pymodules/python2.6/pyPdf/pdf.py:52: DeprecationWarning: the sets module is deprecated
  from sets import ImmutableSet
/usr/lib/pymodules/python2.6/graphviz/graphviz.py:23: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Traceback (most recent call last):
  File "/usr/bin/trac-admin", line 9, in <module>
    load_entry_point('Trac==0.11.7', 'console_scripts', 'trac-admin')()
  File "/usr/lib/python2.6/dist-packages/trac/admin/console.py", line 1321, in run
    return admin.onecmd(command)
  File "/usr/lib/python2.6/dist-packages/trac/admin/console.py", line 138, in onecmd
    rv = cmd.Cmd.onecmd(self, line) or 0
  File "/usr/lib/python2.6/cmd.py", line 219, in onecmd
    return func(arg)
  File "/usr/lib/python2.6/dist-packages/trac/admin/console.py", line 1159, in do_upgrade
    self.__env.upgrade(backup=do_backup)
  File "/usr/lib/python2.6/dist-packages/trac/env.py", line 464, in upgrade
    participant.upgrade_environment(db)
  File "/usr/local/lib/python2.6/dist-packages/tracdownloads/init.py", line 38, in upgrade_environment
    module.do_upgrade(self.env, cursor)
  File "/usr/local/lib/python2.6/dist-packages/tracdownloads/db/db1.py", line 62, in do_upgrade
    cursor.execute(statement)
  File "/usr/lib/python2.6/dist-packages/trac/db/util.py", line 65, in execute
    return self.cursor.execute(sql)
  File "/usr/lib/python2.6/dist-packages/trac/db/sqlite_backend.py", line 80, in execute
    result = PyFormatCursor.execute(self, *args)
  File "/usr/lib/python2.6/dist-packages/trac/db/sqlite_backend.py", line 59, in execute
    args or [])
  File "/usr/lib/python2.6/dist-packages/trac/db/sqlite_backend.py", line 51, in _rollback_on_error
    return function(self, *args, **kwargs)
sqlite3.OperationalError: table download already exists

If I manually inspect the sqlite DB it is clear that no download tables or entries exist before the upgrade. After the first upgrade the download table exists but the download_version entry is missing, which leads to the failed second upgrade.

A possible workaround is to wrap the do_upgrade SQL operations in try/except and continue on errors, but maybe it should really be fixed instead.

http://trac-hacks.org/ticket/4356 looks similar, so maybe the rollback solution applies here as well. I'm not familiar with the Trac internals, however, so I attach a simple diff against db/db1.py to hacks around the issue. Feel free to use it if you like.

Logs also attached with more details for the three combinations of plugins enabled/disabled and with the workaround.

The logs show that it works if the plugins are enabled only one at a time with upgrades in between or if the workaround is applied. The latter prints the

Warning: could not create tables - maybe they exist?

message and continues as expected.

I still have the pristine sqlite-xyz.bak files and trac.ini files that I can provide if that is any help.


Downloadsplugin: 0.11 branch (r11035, pip installed from trac-hacks.org)
TracPasteplugin: 0.11 branch (r11035, pip installed from trac-hacks.org)
Trac: 0.11.7-4 debian package
Server: Apache 2.2.16-6+squeeze4 debian package
Database: sqlite 3.7.3-1 debian package
OS: Linux

Attachments (5)

trac-failed-upgrade.log (24.0 KB) - added by Jonas Bardino 6 years ago.
failing upgrade with both plugins enabled
trac-downloads-only-upgrade.log (21.0 KB) - added by Jonas Bardino 6 years ago.
Working upgrade with only DownloadsPlugin enabled
trac-pastebin-only-upgrade.log (23.5 KB) - added by Jonas Bardino 6 years ago.
Working upgrade with only TracPastePlugin enabled
trac-patched-upgrade.log (34.5 KB) - added by Jonas Bardino 6 years ago.
Working upgrade with both plugins and the workaround enabled
db1.py.diff (1.1 KB) - added by Jonas Bardino 6 years ago.
Patch for db/db1.py to workaround inconsistent downloads table state

Download all attachments as: .zip

Change History (10)

Changed 6 years ago by Jonas Bardino

Attachment: trac-failed-upgrade.log added

failing upgrade with both plugins enabled

Changed 6 years ago by Jonas Bardino

Working upgrade with only DownloadsPlugin enabled

Changed 6 years ago by Jonas Bardino

Working upgrade with only TracPastePlugin enabled

Changed 6 years ago by Jonas Bardino

Attachment: trac-patched-upgrade.log added

Working upgrade with both plugins and the workaround enabled

Changed 6 years ago by Jonas Bardino

Attachment: db1.py.diff added

Patch for db/db1.py to workaround inconsistent downloads table state

comment:1 Changed 6 years ago by Radek Bartoň

Status: newassigned

Thank you for reporting this. Adding a rollback on SQL exception is certainly a good idea, although it is not clear to me why the error occurs only when both plugins are enabled. There is no obvious reason why should they collide.

comment:2 Changed 6 years ago by Jonas Bardino

Further investigations indicate that the upgrades actually run in parallel and that after DownloadsPlugin has finished upgrading with all the DB changes, they are visible initially from the TracPastePlugin and then downloads_version suddenly disappears for no obvious reason.

I tried shoveling in debug messages throughout the upgrade processes and the immediate call to _get_version() in tracpaste/db.py (which only executes sql SELECTs) results in downloads_version disappearing. To be precise it is the last part of _get_version() counting existing 'pastes' that fails and somehow removes downloads_version. The 'pastes' table is not there the first time so the SELECT throws an expected 'no such table: pastes' exception. I would assume that it would be enough to disable the explicit db.rollback() in the exception handler there, but that doesn't help. However, completely removing that try/except section seems to solve the problem.

I actually discovered that an explicit db.commit() at the end of upgrade_environment in tracdownloads/init.py also prevents the disappearance of downloads_version, but AFAICT from the Trac documentation, that should not be needed. This thread may be relevant, though: http://old.nabble.com/Environment.upgrade_environment-and-commits-td28364661.html

Thanks for your efforts!

Cheers, Jonas

comment:3 Changed 6 years ago by Radek Bartoň

On a contrary, thank you for your effort. You saved me a lot of time investigating the cause.

comment:4 Changed 4 years ago by Ryan J Ollos

Owner: changed from Radek Bartoň to Ryan J Ollos

comment:5 Changed 12 months ago by Ryan J Ollos

Resolution: worksforme
Status: assignedclosed

I think you won't see this issue in Trac 1.0 and the latest versions of both plugins, but feel free to reopen if you can reproduce with the latest.

Last edited 12 months ago by Ryan J Ollos (previous) (diff)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.