Update installation 1.4.6 -> 1.4.7 not possible

Hi, I'll take a look asap.

In the meantime:

• I noticed you are using the test manager egg files for Python 2.7 while all other egg files are for Python 2.6. It doesn't seem to create any problem so far, but I suggest that you use the correct egg files for you Python version.
• Which database are you using?

Ciao, Roberto

Hi, I've tried the upgrade from 1.4.6 to 1.4.7 and it does work in my env:

• Python 2.7
• Trac 0.12.2
• sqlite

Some more info:

• I upgraded using the admin page, running the upgrade db commands from a command line,
• I restarted Trac after the installation of TracGenericClass?

Please, answer the questions above and let me know.

Ciao, Roberto

Hello Roberto,

I've already removed the 2.7 eggs and compieled 2.6 eggs - but with the same installation problems. I've also done the needed Trac restarts.

But I'm using postgres 8.4 - I think your CREATE TABLE "testmanager_templates" is not working with postgres :-( ...

look at debugging logfile ...

2011-08-31 13:10:27,278 Trac[default_workflow] WARNING: Ticket workflow action 'mylyn' doesn't define any transitions
2011-08-31 13:10:27,279 Trac[util] DEBUG: SQL: "SELECT value FROM system WHERE name='database_version'"
2011-08-31 13:10:27,282 Trac[util] DEBUG: SQL: 'select count(*) from testcase'
2011-08-31 13:10:27,283 Trac[util] DEBUG: SQL: 'select count(*) from testplan'
2011-08-31 13:10:27,284 Trac[util] DEBUG: SQL: 'select count(*) from testconfig'
2011-08-31 13:10:27,285 Trac[util] DEBUG: SQL: 'select count(*) from testcaseinplan'
2011-08-31 13:10:27,286 Trac[util] DEBUG: SQL: 'select count(*) from testcasehistory'
2011-08-31 13:10:27,287 Trac[util] DEBUG: SQL: 'select count(*) from testmanager_templates'
2011-08-31 13:10:27,288 Trac[util] DEBUG: execute exception: ProgrammingError('relation "testmanager_templates" does not exist\nLINE 1: select count(*) from testmanager_templates\n                             ^\n',)
2011-08-31 13:10:27,289 Trac[model] DEBUG: Need to create db tables for class 'testmanager_templates'.
2011-08-31 13:10:27,289 Trac[env] WARNING: Component <testmanager.model.TestManagerModelProvider object at 0xa6fcd6c> requires environment upgrade
2011-08-31 13:10:27,289 Trac[util] DEBUG: SQL: "SELECT value FROM system WHERE name='database_version'"
2011-08-31 13:10:27,291 Trac[util] DEBUG: SQL: 'select count(*) from testcase'
2011-08-31 13:10:27,292 Trac[util] DEBUG: SQL: 'select count(*) from testplan'
2011-08-31 13:10:27,292 Trac[util] DEBUG: SQL: 'select count(*) from testconfig'
2011-08-31 13:10:27,293 Trac[util] DEBUG: SQL: 'select count(*) from testcaseinplan'
2011-08-31 13:10:27,294 Trac[util] DEBUG: SQL: 'select count(*) from testcasehistory'
2011-08-31 13:10:27,294 Trac[util] DEBUG: SQL: 'select count(*) from testmanager_templates'
2011-08-31 13:10:27,295 Trac[util] DEBUG: execute exception: ProgrammingError('relation "testmanager_templates" does not exist\nLINE 1: select count(*) from testmanager_templates\n                             ^\n',)
2011-08-31 13:10:27,296 Trac[model] DEBUG: Need to create db tables for class 'testmanager_templates'.
2011-08-31 13:10:27,296 Trac[util] DEBUG: SQL: 'select count(*) from resourceworkflowstate'
2011-08-31 13:10:27,297 Trac[env] INFO: testmanager.model.TestManagerModelProvider upgrading...
2011-08-31 13:10:27,297 Trac[util] DEBUG: SQL: 'select count(*) from testcase'
2011-08-31 13:10:27,298 Trac[util] DEBUG: SQL: 'select count(*) from testplan'
2011-08-31 13:10:27,299 Trac[util] DEBUG: SQL: 'select count(*) from testconfig'
2011-08-31 13:10:27,299 Trac[util] DEBUG: SQL: 'select count(*) from testcaseinplan'
2011-08-31 13:10:27,300 Trac[util] DEBUG: SQL: 'select count(*) from testcasehistory'
2011-08-31 13:10:27,301 Trac[util] DEBUG: SQL: 'select count(*) from testmanager_templates'
2011-08-31 13:10:27,301 Trac[util] DEBUG: execute exception: ProgrammingError('relation "testmanager_templates" does not exist\nLINE 1: select count(*) from testmanager_templates\n                             ^\n',)
2011-08-31 13:10:27,302 Trac[model] DEBUG: Need to create db tables for class 'testmanager_templates'.
2011-08-31 13:10:27,302 Trac[model] INFO: Upgrading DB...
2011-08-31 13:10:27,303 Trac[util] DEBUG: SQL: 'select count(*) from testcase'
2011-08-31 13:10:27,304 Trac[util] DEBUG: SQL: 'select count(*) from testplan'
2011-08-31 13:10:27,304 Trac[util] DEBUG: SQL: 'select count(*) from testconfig'
2011-08-31 13:10:27,305 Trac[util] DEBUG: SQL: 'select count(*) from testcaseinplan'
2011-08-31 13:10:27,306 Trac[util] DEBUG: SQL: 'select count(*) from testcasehistory'
2011-08-31 13:10:27,306 Trac[util] DEBUG: SQL: 'select count(*) from testmanager_templates'
2011-08-31 13:10:27,307 Trac[util] DEBUG: execute exception: ProgrammingError('relation "testmanager_templates" does not exist\nLINE 1: select count(*) from testmanager_templates\n                             ^\n',)
2011-08-31 13:10:27,307 Trac[model] INFO: Creating base table testmanager_templates...
2011-08-31 13:10:27,307 Trac[model] DEBUG: CREATE TABLE "testmanager_templates" (
    "id" text,
    "name" text,
    "type" text,
    "description" text,
    "content" text,
    CONSTRAINT "testmanager_templates_pk" PRIMARY KEY ("id","name","type")
)
2011-08-31 13:10:27,308 Trac[util] DEBUG: SQL: 'CREATE TABLE "testmanager_templates" (\n    "id" text,\n    "name" text,\n    "type" text,\n    "description" text,\n    "content" text,\n    CONSTRAINT "testmanager_templates_pk" PRIMARY KEY ("id","name","type")\n)'
2011-08-31 13:10:27,308 Trac[util] DEBUG: execute exception: InternalError('current transaction is aborted, commands ignored until end of transaction block\n',)
2011-08-31 13:10:27,311 Trac[model] ERROR: ('InternalError', '<no args>', ['  File "build/bdist.linux-i686/egg/tracgenericclass/model.py", line 1286, in upgrade_db\n    cursor.execute(stmt)\n', '  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/util.py", line 56, in execute\n    r = self.cursor.execute(sql)\n'])
2011-08-31 13:10:27,311 Trac[model] ERROR: Exception during database creation.
2011-08-31 13:10:27,315 Trac[console] ERROR: Exception in trac-admin command:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/console.py", line 107, in onecmd
    rv = cmd.Cmd.onecmd(self, line) or 0
  File "/usr/lib/python2.6/cmd.py", line 218, in onecmd
    return self.default(line)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/console.py", line 266, in default
    return cmd_mgr.execute_command(*args)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/api.py", line 123, in execute_command
    return f(*fargs)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/env.py", line 782, in _do_upgrade
    self.env.upgrade(backup=no_backup is None)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/env.py", line 525, in upgrade
    with_transaction(self)(participant.upgrade_environment)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/api.py", line 77, in transaction_wrapper
    fn(ldb)
  File "build/bdist.linux-i686/egg/testmanager/model.py", line 649, in upgrade_environment
    upgrade_db(self.env, self.SCHEMA, db)
  File "build/bdist.linux-i686/egg/tracgenericclass/model.py", line 1286, in upgrade_db
    cursor.execute(stmt)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/util.py", line 56, in execute
    r = self.cursor.execute(sql)
InternalError: current transaction is aborted, commands ignored until end of transaction block
InternalError: current transaction is aborted, commands ignored until end of transaction block

look at the \n in the logfile - maby this can be a problem ? because this \n are missing at CREATE TABLE "testmanager_templates"

Now I've tried to create a complete new trac project ...

Install TracGenericClass? + restart trac = OK Install TracGenericWorkflow? + "trac-admin /data/trac/testm upgrade --no-backup" = ERROR

[root@atpc03nd testman4trac.1.4.7]# tail -50 /data/trac/testm/log/trac.log
2011-09-01 10:14:57,413 Trac[loader] DEBUG: Loading tracgenericclass from /data/trac/testm/plugins/TracGenericClass-1.0.7-py2.6.egg
2011-09-01 10:14:57,418 Trac[loader] DEBUG: Loading tracgenericworkflow from /data/trac/testm/plugins/TracGenericWorkflow-1.0.2-py2.6.egg
2011-09-01 10:14:57,421 Trac[loader] DEBUG: Loading tracopt.mimeview.enscript from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,423 Trac[loader] DEBUG: Loading tracopt.mimeview.php from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,425 Trac[loader] DEBUG: Loading tracopt.perm.authz_policy from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,427 Trac[loader] DEBUG: Loading tracopt.perm.config_perm_provider from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,428 Trac[loader] DEBUG: Loading tracopt.ticket.commit_updater from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,431 Trac[loader] DEBUG: Loading tracopt.ticket.deleter from /usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg
2011-09-01 10:14:57,454 Trac[default_workflow] DEBUG: Workflow actions at initialization: {u'resolve': {u'operations': [u'set_resolution'], 'name': u'resolve', 'default': 0, 'newstate': u'closed', 'oldstates': [u'new', u'assigned', u'accepted', u'reopened'], u'permissions': [u'TICKET_MODIFY']}, u'accept': {u'operations': [u'set_owner_to_self'], 'name': u'accept', 'default': 0, 'newstate': u'accepted', 'oldstates': [u'new', u'assigned', u'accepted', u'reopened'], u'permissions': [u'TICKET_MODIFY']}, u'leave': {u'operations': [u'leave_status'], 'name': u'leave', u'default': 1, 'newstate': u'*', 'oldstates': [u'*'], 'permissions': []}, u'reopen': {u'operations': [u'del_resolution'], 'name': u'reopen', 'default': 0, 'newstate': u'reopened', 'oldstates': [u'closed'], u'permissions': [u'TICKET_CREATE']}, u'reassign': {u'operations': [u'set_owner'], 'name': u'reassign', 'default': 0, 'newstate': u'assigned', 'oldstates': [u'new', u'assigned', u'accepted', u'reopened'], u'permissions': [u'TICKET_MODIFY']}, '_reset': {'operations': ['reset_workflow'], 'name': 'reset', 'default': 0, 'newstate': 'new', 'oldstates': [], 'permissions': []}}

2011-09-01 10:14:57,458 Trac[model] DEBUG: Need to create db tables for class 'resourceworkflowstate'.
2011-09-01 10:14:57,458 Trac[env] WARNING: Component <tracgenericworkflow.model.GenericWorkflowModelProvider object at 0x9e3890c> requires environment upgrade
2011-09-01 10:14:57,460 Trac[model] DEBUG: Need to create db tables for class 'resourceworkflowstate'.
2011-09-01 10:14:57,461 Trac[env] INFO: tracgenericworkflow.model.GenericWorkflowModelProvider upgrading...
2011-09-01 10:14:57,462 Trac[model] DEBUG: Need to create db tables for class 'resourceworkflowstate'.
2011-09-01 10:14:57,463 Trac[model] INFO: Upgrading DB...
2011-09-01 10:14:57,464 Trac[model] INFO: Creating base table resourceworkflowstate...
2011-09-01 10:14:57,464 Trac[model] DEBUG: CREATE TABLE "resourceworkflowstate" (
    "id" text,
    "res_realm" text,
    "state" text,
    CONSTRAINT "resourceworkflowstate_pk" PRIMARY KEY ("id","res_realm")
)
2011-09-01 10:14:57,467 Trac[model] ERROR: ('InternalError', '<no args>', ['  File "build/bdist.linux-i686/egg/tracgenericclass/model.py", line 1286, in upgrade_db\n    cursor.execute(stmt)\n', '  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/util.py", line 66, in execute\n    return self.cursor.execute(sql)\n'])
2011-09-01 10:14:57,467 Trac[model] ERROR: Exception during database creation.
2011-09-01 10:14:57,471 Trac[console] ERROR: Exception in trac-admin command:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/console.py", line 107, in onecmd
    rv = cmd.Cmd.onecmd(self, line) or 0
  File "/usr/lib/python2.6/cmd.py", line 218, in onecmd
    return self.default(line)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/console.py", line 266, in default
    return cmd_mgr.execute_command(*args)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/admin/api.py", line 123, in execute_command
    return f(*fargs)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/env.py", line 782, in _do_upgrade
    self.env.upgrade(backup=no_backup is None)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/env.py", line 525, in upgrade
    with_transaction(self)(participant.upgrade_environment)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/api.py", line 77, in transaction_wrapper
    fn(ldb)
  File "build/bdist.linux-i686/egg/tracgenericworkflow/model.py", line 138, in upgrade_environment
    upgrade_db(self.env, self.SCHEMA, db)
  File "build/bdist.linux-i686/egg/tracgenericclass/model.py", line 1286, in upgrade_db
    cursor.execute(stmt)
  File "/usr/lib/python2.6/site-packages/Trac-0.12.2-py2.6.egg/trac/db/util.py", line 66, in execute
    return self.cursor.execute(sql)
InternalError: current transaction is aborted, commands ignored until end of transaction block
InternalError: current transaction is aborted, commands ignored until end of transaction block

look at your model.py !

After a deep look into your sources, I've found the possible mistakes !

Maybe you've forgotten to define the "FILEDS" types (like: text) for the new tables ...

Maby "METADATA" is also incomplete ...

Hi Roberto, hi Andreas,

it's not a problem of the create-statement. It's the same we've (Robert an I) already talked before: creating the tables run within a "transaction block" and the problem (at least with postgresql) is, that when checking whether the table exists ("select count from...") an exception is thrown that is not handled! and then the following statement "create table..." will fail as postgresql expects this exception to be handled!

As already written in ticket #8907 (comment:17) would solve this problem, but (as also stated) this is definitly not the best way to do the updates...

Christian

Hi Andreas and Christian, yes, what Christian says is right, the problem is how I check whether a db upgrade is necessary.

I think a better solution would be to do like Trac does, so to keep track of the current database version in some configuration table, and check if it is current or old.

In case a database upgrade is necessary, a series of modules would be called each handling a single step of the upgrade from db version x to db version x+1, until the desired version y.

This work requires some time, and I'm a littlebit busy with my job at the moment.

In the meantime, the temporary solution that Christian suggested in #8907 would solve your problem, Andreas.

I'll try to find some time tonight or tomorrow to make a patch and attach it here.

Thanks, Christian, for your help! Stay tuned :D Ciao, Roberto

Patch to fix the problem

Hi Andreas, you will find in attachment a patch that fixes the problem temporarily as suggested by Christian.

Unzip the archive and replace the files with the same names in the 1.4.7 source tree, then build and replace the produced egg files into your Trac plugins directory.

Let me know how it goes.

Ciao, Roberto

The upgrade in an existing 1.4.6 was now OK.

But installing the plugin in a new trac project failed !!'''

# trac-admin /data/trac/testm2 upgrade --no-backup

ProgrammingError: relation "testconfig" does not exist
LINE 1: SELECT COUNT(*) FROM testconfig WHERE propname=E'NEXT_CATALO...

After some ...

# service httpd restart
# trac-admin /data/trac/testm2 upgrade --no-backup

... also this new trac project with your patches was OK

Crreated a NEW project ...

# service httpd restart

TracGenericClass?:

# service httpd restart

TracGenericWorkflow?:

# trac-admin /data/trac/testm3 upgrade --no-backup
# service httpd restart

TestManager?:

# trac-admin /data/trac/testm3 upgrade --no-backup
ProgrammingError: relation "testconfig" does not exist
LINE 1: SELECT COUNT(*) FROM testconfig WHERE propname=E'NEXT_CATALO...
# service httpd restart
# trac-admin /data/trac/testm3 upgrade --no-backup
ProgrammingError: relation "testconfig" does not exist
LINE 1: SELECT COUNT(*) FROM testconfig WHERE propname=E'NEXT_CATALO...
# service httpd restart
# trac-admin /data/trac/testm3 upgrade --no-backup
ProgrammingError: relation "testconfig" does not exist
LINE 1: SELECT COUNT(*) FROM testconfig WHERE propname=E'NEXT_CATALO...
# service httpd restart
# trac-admin /data/trac/testm3 upgrade --no-backup
Upgrade done.

A quick solution would be to move the "final db.commit" (in upgrade_db at line 1341) into the for-loop ("for realm in schema:").

I know, this is not a perfect soltion, but at least would work. Your current patch does the following:

If a table is missing, then a rollback will be done (that's ok). Then the table(s) will be created. After that, the next check for a table runs and probably also failes (if it's not existing) and then again a rollback will be done. And exactly this rollback will also rollback the creation of the table(s) before...

So, you're completely right when saying that the best solution would be doing the upgrade the same way as any other plugins (and also trac itself) are doing... :-)

Christian

Hi, I found some time to work on this plugin today and I have attached a development 1.4.8 version to ticket #9208 that also contains a rewritten, more robust installation and upgrade mechanism, derived from the Trac one.

It only upgrades from 1.4.7 to 1.4.8, it does not work as an upgrade form previous versions, anyway if you could try it out on PostgreSQL it would help.

You may also want to take a look at this post and maybe give your opinion.

Please, let me know if the basic installation and the upgrade from 1.4.7 work with the 1.4.8 dev version.

Ciao, Roberto

I've tested the installation of your 1.4.8 plugin:

from > to > result

1.4.6 > 1.4.8 = update installation ok
1.4.7 > 1.4.8 = update installation ok
none  > 1.4.8 = initial installation ok

Ciao, Andreas

Ciao Andreas, thanks for your testing! Actually I'm surprised the upgrade 1.4.6 -> 1.4.8 worked fine, I suspect it may have just exited without doing nothing and not giving errors. I was just wondering if you did a little check whether the new test plan functionality was actually there or not. Anyway I'll try to do it as soon as I can find some time (my job is really killing me these months...)

Ciao, Roberto

Fixed with 1.4.8.