MySQL to PostgreSQL migration fails with worklog usermanual

[Quince]

This is a great plugin that we use to slave-drive oversee our devs. Unfortunately, it seems it can't be migrated with the TracMigratePlugin:

./trac-migrate.py --in-place /srv/trac postgres://tracuser:nalkyergatte@127.0.0.1/trac?schema=trac

Worklog needs an upgrade
 * Upgrading Database
Creating work_log table
Updating work_log table (v2)
Updating work_log table (v3)
 * Upgrading usermanual
Traceback (most recent call last):
  File "./trac-migrate.py", line 55, in <module>
    sys.exit(main(sys.argv[1:]) or 0)
  File "./trac-migrate.py", line 51, in main
    return TracMigrationCommand(env)._do_migrate(dest, dburi)
  File "/tmp/m/tracmigrate/admin.py", line 29, in _do_migrate
    return self._do_migrate_inplace(dburi)
  File "/tmp/m/tracmigrate/admin.py", line 60, in _do_migrate_inplace
    env = self._create_env(env_path, dburi)
  File "/tmp/m/tracmigrate/admin.py", line 92, in _create_env
    env = Environment(env_path, create=True, options=options)
  File "/usr/lib/python2.7/site-packages/trac/core.py", line 124, in __call__
    self.__init__(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/trac/env.py", line 288, in __init__
    setup_participant.environment_created()
  File "build/bdist.linux-x86_64/egg/worklog/api.py", line 55, in environment_created
  File "build/bdist.linux-x86_64/egg/worklog/api.py", line 182, in upgrade_environment
  File "build/bdist.linux-x86_64/egg/worklog/api.py", line 146, in do_user_man_update
AttributeError: 'NoneType' object has no attribute 'cursor'

[Jun Omae]

I've improved TracMigratePlugin in [13808]. Please try the latest of TracMigratePlugin. I think the migration currently is robust.

I consider that most of plugins which use IEnvironmentSetupParticipant have not been tested on initenv....

Also, proposed changes for the original issue.

worklogplugin/trunk/worklog/api.py

@@ -51 +51 @@
 
     def environment_created(self):
         """Called when a new Trac environment is created."""
-        if self.environment_needs_upgrade(None):
-            self.upgrade_environment(None)
+        @self.env.with_transaction()
+        def tx(db):
+            self.upgrade_environment(db)
 
     def system_needs_upgrade(self):
         return self.db_installed_version < self.db_version

After the changes

$ cat inherit.ini
[components]
worklog = enabled
$ PYTHONPATH=. ~/venv/trac/0.12.5/bin/trac-admin /dev/shm/th11670 initenv --inherit=$PWD/inherit.ini th11670 sqlite:db/trac.db > /dev/null
$ echo $?
0
$ sqlite3 /dev/shm/th11670/db/trac.db '.schema work_log'
CREATE TABLE work_log (worker     TEXT,ticket     INTEGER,lastchange INTEGER,starttime  INTEGER,endtime    INTEGER, comment TEXT);

[anonymous]

Hi jun66j5

I made this change in api.py and got the updated TracMigratePlugin. I got no errors when I ran the migrate command from trac-admin, but then, when I go to the Work Log tab in trac, I see this:

Trac detected an internal error:

ProgrammingError: relation "work_log" does not exist
LINE 1: ...(SELECT worker,MAX(lastchange) AS lastchange FROM work_log G...
                                                             ^

The migration seems to not have copied the work_log table even though the plugin is enabled in trac.ini. I'm guessing I need to execute the "After the changes" commands you listed above, but my current database is MySQL, not SQLite, and I don't know how to change the commands to MySQL format.

[Quince]

It seems to work after I reverted the changes to api.py...

[Jun Omae]

Hmm, worklogplugin with the patch and the latest of tracmigrateplugin work for me.

$ grep database /dev/shm/th11670-test/conf/trac.ini
database = mysql://tracuser:password@localhost/trac_worklog
$ ~/venv/trac/1.0.1/bin/python trac-migrate.py --in-place /dev/shm/th11670-test postgres://tracuser:password@localhost/tractest?schema=worklog
Worklog needs an upgrade
 * Upgrading Database
Creating work_log table
Updating work_log table (v2)
Updating work_log table (v3)
 * Upgrading usermanual
Done upgrading Worklog
Copying tables:
  attachment table... 0 records.
  auth_cookie table... 0 records.
  cache table... 1 records.
  component table... 2 records.
  enum table... 13 records.
  milestone table... 4 records.
  node_change table... 0 records.
  permission table... 16 records.
  report table... 8 records.
  repository table... 0 records.
  revision table... 0 records.
  session table... 0 records.
  session_attribute table... 0 records.
  system table... 3 records.
  ticket table... 0 records.
  ticket_change table... 0 records.
  ticket_custom table... 0 records.
  version table... 2 records.
  wiki table... 59 records.
  work_log table... 0 records.
Back up conf/trac.ini to conf/trac.ini.migrate-1396581184 in /dev/shm/th11670-test.
$ sudo -u postgres psql
Welcome to psql 8.1.23, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit

postgres=# \c tractest
You are now connected to database "tractest".
tractest=# set search_path = worklog,public;
SET
tractest=# \d
                 List of relations
 Schema  |       Name        |   Type   |  Owner
---------+-------------------+----------+----------
 worklog | attachment        | table    | tracuser
 worklog | auth_cookie       | table    | tracuser
...
 worklog | wiki              | table    | tracuser
 worklog | work_log          | table    | tracuser
(22 rows)

[Ryan J Ollos]

Replying to jun66j5:

Also, proposed changes for the original issue.

Feel free to push any changes for the WorkLogPlugin. coling has previously given me the go-ahead to push any contributions.

[Jun Omae]

In 13835:

WorkLogPlugin: fixed environment_created() not working on initenv (refs #11669)