Unable to upgrade OS X tracd 0.11.7 database

[Sam Halliday]

I get the following exception when attempting to upgrade the TRAC install, with a similar thing when I go to any issue:

$ trac-admin /Users/samuel/Desktop/TRAC/test upgrade

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/trac-admin", line 8, in <module>
    load_entry_point('Trac==0.11.7', 'console_scripts', 'trac-admin')()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/admin/console.py", line 1321, in run
    return admin.onecmd(command)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/admin/console.py", line 138, in onecmd
    rv = cmd.Cmd.onecmd(self, line) or 0
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/admin/console.py", line 1154, in do_upgrade
    if not self.__env.needs_upgrade():
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/env.py", line 435, in needs_upgrade
    for participant in self.setup_participants:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/core.py", line 70, in extensions
    return filter(None, [component.compmgr[cls] for cls in extensions])
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/core.py", line 207, in __getitem__
    component = cls(self)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Trac-0.11.7-py2.7.egg/trac/core.py", line 111, in maybe_init
    init(self)
  File "build/bdist.macosx-10.8-x86_64/egg/worklog/api.py", line 36, in __init__
AttributeError: 'Environment' object has no attribute 'get_read_db'

[Sam Halliday]

I went back to r10794, 0.11 branch, and this works ok.

[Ryan J Ollos]

In [11830] I renamed 0.12 to trunk and made the comment that the 0.11 and 0.12 branches were equivalent. Looking at it now, however, I see that is not the case. I must have done something stupid, like diff 0.11 0.12 without the -r flag. I'll fix this now.

[Ryan J Ollos]

Here is the 0.11 and 0.12 diff @ r11829:

setup.py

@@ -7 +7 @@
 setup(name=PACKAGE,
       description='Plugin to manage the which tickets users are currently working on',
       keywords='trac plugin ticket working',
-      version='0.1',
+      version='0.2',
       url='',
       license='http://www.opensource.org/licenses/mit-license.php',
       author='Colin Guthrie',

worklog/api.py

@@ -33 +33 @@
         self.db_installed_version = None
 
         # Initialise database schema version tracking.
-        db = self.env.get_db_cnx()
+        db = self.env.get_read_db()
         cursor = db.cursor()
         cursor.execute("SELECT value FROM system WHERE name=%s", (self.db_version_key,))
         try:
             self.db_installed_version = int(cursor.fetchone()[0])
         except:
-            self.db_installed_version = 0
-            try:
-                cursor.execute("INSERT INTO system (name,value) VALUES(%s,%s)",
-                               (self.db_version_key, self.db_installed_version))
-                db.commit()
-            except Exception, e:
-                db.rollback()
-                raise e
-
+            @self.env.with_transaction()
+            def do_init(db):
+              cursor = db.cursor()
+              self.db_installed_version = 0
+              cursor.execute("INSERT INTO system (name,value) VALUES(%s,%s)",
+                             (self.db_version_key, self.db_installed_version))
     
     def environment_created(self):
         """Called when a new Trac environment is created."""
@@ -57 +54 @@
     def system_needs_upgrade(self):
         return self.db_installed_version < self.db_version
 
-    def do_db_upgrade(self):
+    def do_db_upgrade(self, db):
         # Legacy support hack (supports upgrades from revisions r2495 or before)
-        db = self.env.get_db_cnx()
-        cursor = db.cursor()
         if self.db_installed_version == 0:
             try:
+                cursor = self.env.get_read_db().cursor()
                 cursor.execute('SELECT * FROM work_log LIMIT 1')
-                db.commit()
                 # We've succeeded so we actually have version 1
                 self.db_installed_version = 1
             except:
-                db.rollback()
+                cursor.connection.rollback()
+                self.db_installed_version = 0
         # End Legacy support hack
 
         # Do the staged updates
@@ -123 +119 @@
             # Updates complete, set the version
             cursor.execute("UPDATE system SET value=%s WHERE name=%s", 
                            (self.db_version, self.db_version_key))
-            db.commit()
         except Exception, e:
             self.log.error("WorklogPlugin Exception: %s" % (e,));
             db.rollback()
@@ -132 +127 @@
 
     
     def needs_user_man(self):
-        db = self.env.get_db_cnx()
+        db = self.env.get_read_db()
         cursor = db.cursor()
         try:
             cursor.execute('SELECT MAX(version) FROM wiki WHERE name=%s', (user_manual_wiki_title,))
-            db.commit()
             maxversion = int(cursor.fetchone()[0])
         except:
-            db.rollback()
+            cursor.connection.rollback()
             maxversion = 0
 
         return maxversion < user_manual_version
 
-    def do_user_man_update(self):
+    def do_user_man_update(self, db):
         when = int(time.time())
-        db = self.env.get_db_cnx()
         cursor = db.cursor()
         try:
             cursor.execute('INSERT INTO wiki (name,version,time,author,ipnr,text,comment,readonly) '
@@ -176 +169 @@
         transactions. This is done implicitly after all participants have
         performed the upgrades they need without an error being raised.
         """
-        def p(s):
-            print s
-            return True
         print "Worklog needs an upgrade"
         if self.system_needs_upgrade():
-            p("Upgrading Database")
-            self.do_db_upgrade()
+            print " * Upgrading Database"
+            @self.env.with_transaction(db)
+            def real_do_db_upgrade(db):
+              self.do_db_upgrade(db)
         if self.needs_user_man():
-            p("Upgrading usermanual")
-            self.do_user_man_update()
+            print " * Upgrading usermanual"
+            @self.env.with_transaction(db)
+            def real_do_user_man_update(db):
+              self.do_user_man_update(db)
         print "Done upgrading Worklog"
 
 

worklog/manager.py

@@ -65 +65 @@
         # If we get here then we know we can start work :)
         return True
 
-    def save_ticket(self, tckt, db, msg):
+    def save_ticket(self, tckt, msg):
         # determine sequence number... 
         cnum = 0
         tm = TicketModule(self.env)
-        for change in tm.grouped_changelog_entries(tckt, db):
+        for change in tm.grouped_changelog_entries(tckt, None):
             if change['permanent']:
                 cnum += 1
         nowdt = self.now
         nowdt = to_datetime(nowdt)
-        tckt.save_changes(self.authname, msg, nowdt, db, cnum+1)
+        tckt.save_changes(self.authname, msg, nowdt, None, cnum+1)
         ## Often the time overlaps and causes a db error,
         ## especially when the trac integration post-commit hook is used.
         ## NOTE TO SELF. I DON'T THINK THIS IS NECESSARY RIGHT NOW...
         #count = 0
         #while count < 10:
         #    try:
-        #        tckt.save_changes(self.authname, msg, self.now, db, cnum+1)
+        #        tckt.save_changes(self.authname, msg, self.now, None, cnum+1)
         #        count = 42
         #    except Exception, e:
         #        self.now += 1
         #        count += 1
-        db.commit()
         
         tn = TicketNotifyEmail(self.env)
         tn.notify(tckt, newticket=0, modtime=nowdt)
@@ -104 +103 @@
         # ticket modification.
 
         # If the ticket is closed, we need to reopen it.
-        db = self.env.get_db_cnx()
-        tckt = Ticket(self.env, ticket, db)
+        tckt = Ticket(self.env, ticket)
 
         if 'closed' == tckt['status']:
             tckt['status'] = 'reopened'
             tckt['resolution'] = ''
-            self.save_ticket(tckt, db, 'Automatically reopening in order to start work.')
+            self.save_ticket(tckt, 'Automatically reopening in order to start work.')
 
             # Reinitialise for next test
-            db = self.env.get_db_cnx()
-            tckt = Ticket(self.env, ticket, db)
+            tckt = Ticket(self.env, ticket)
 
             
         if self.authname != tckt['owner']:
@@ -123 +120 @@
                 tckt['status'] = 'accepted'
             else:
                 tckt['status'] = 'new'
-            self.save_ticket(tckt, db, 'Automatically reassigning in order to start work.')
+            self.save_ticket(tckt, 'Automatically reassigning in order to start work.')
 
             # Reinitialise for next test
-            db = self.env.get_db_cnx()
-            tckt = Ticket(self.env, ticket, db)
+            tckt = Ticket(self.env, ticket)
 
 
         if 'accepted' != tckt['status']:
             tckt['status'] = 'accepted'
-            self.save_ticket(tckt, db, 'Automatically accepting in order to start work.')
+            self.save_ticket(tckt, 'Automatically accepting in order to start work.')
 
         # There is a chance the user may be working on another ticket at the moment
         # depending on config options
         if self.config.getbool('worklog', 'autostopstart'):
             # Don't care if this fails, as with these arguments the only failure
             # point is if there is no active task... which is the desired scenario :)
-            self.stop_work()
+            self.stop_work(comment='Stopping work on this ticket to start work on #%s.' % (ticket))
             self.explanation = ''
  
-        cursor = db.cursor()
-        cursor.execute('INSERT INTO work_log (worker, ticket, lastchange, starttime, endtime) '
-                       'VALUES (%s, %s, %s, %s, %s)',
-                       (self.authname, ticket, self.now, self.now, 0))
-        db.commit()
+        @self.env.with_transaction()
+        def do_log(db):
+          cursor = db.cursor()
+          cursor.execute('INSERT INTO work_log (worker, ticket, lastchange, starttime, endtime) '
+                         'VALUES (%s, %s, %s, %s, %s)',
+                         (self.authname, ticket, self.now, self.now, 0))
         return True
 
     
@@ -168 +165 @@
 
         stoptime = float(stoptime)
         
-        db = self.env.get_db_cnx();
-        cursor = db.cursor()
-        cursor.execute('UPDATE work_log '
-                       'SET endtime=%s, lastchange=%s, comment=%s '
-                       'WHERE worker=%s AND lastchange=%s AND endtime=0',
-                       (stoptime, stoptime, comment, self.authname, active['lastchange']))
-        db.commit()
+        @self.env.with_transaction()
+        def do_log(db):
+          cursor = db.cursor()
+          cursor.execute('UPDATE work_log '
+                         'SET endtime=%s, lastchange=%s, comment=%s '
+                         'WHERE worker=%s AND lastchange=%s AND endtime=0',
+                         (stoptime, stoptime, comment, self.authname, active['lastchange']))
 
         plugtne = self.config.getbool('worklog', 'timingandestimation') and self.config.get('ticket-custom', 'hours')
         plughrs = self.config.getbool('worklog', 'trachoursplugin') and self.config.get('ticket-custom', 'totalhours')
@@ -214 +211 @@
             if not message:
                 message = 'Hours recorded automatically by the worklog plugin.'
 
-            db = self.env.get_db_cnx()
-            tckt = Ticket(self.env, active['ticket'], db)
+            tckt = Ticket(self.env, active['ticket'])
 
             if plugtne:
                 tckt['hours'] = hours
-            self.save_ticket(tckt, db, message)
+            self.save_ticket(tckt, message)
             message = ''
 
         if message:
-            db = self.env.get_db_cnx()
-            tckt = Ticket(self.env, active['ticket'], db)
-            self.save_ticket(tckt, db, message)
+            tckt = Ticket(self.env, active['ticket'])
+            self.save_ticket(tckt, message)
 
         return True
 
 
     def who_is_working_on(self, ticket):
-        db = self.env.get_db_cnx()
+        db = self.env.get_read_db()
         cursor = db.cursor()
         cursor.execute('SELECT worker,starttime FROM work_log WHERE ticket=%s AND endtime=0', (ticket,))
         try:
@@ -248 +243 @@
         if self.authname == 'anonymous':
             return None
 
-        db = self.env.get_db_cnx()
+        db = self.env.get_read_db()
         cursor = db.cursor()
         cursor.execute('SELECT MAX(lastchange) FROM work_log WHERE worker=%s', (self.authname,))
         row = cursor.fetchone()
@@ -289 +284 @@
         return task
 
     def get_work_log(self, mode='all'):
-        db = self.env.get_db_cnx()
+        db = self.env.get_read_db()
         cursor = db.cursor()
         if mode == 'user':
             cursor.execute('SELECT wl.worker, s.value, wl.starttime, wl.endtime, wl.ticket, t.summary, t.status, wl.comment '

worklog/timeline_hook.py

@@ -34 +34 @@
             ts_stop = to_timestamp(stop)
 
             ticket_realm = Resource('ticket')
-            db = self.env.get_db_cnx()
+            db = self.env.get_read_db()
             cursor = db.cursor()
 
             cursor.execute("""SELECT wl.worker,wl.ticket,wl.time,wl.starttime,wl.comment,wl.kind,t.summary,t.status,t.resolution,t.type

[Ryan J Ollos]

(In [12089]) Refs #10206, #10251:

• Made the trunk compatible with Trac 0.11, by rolling out changes introduced in [9480] and later. There is nothing forcing us to adapt to the new Trac DB API at the moment, and it is easier to maintain a single code-base that is compatible back to Trac 0.11. The Trac DB API changed again in 1.0, so we are better off waiting and adapting to that API down the road anyway.
• Attempted to cleanup the mess of import statements in every module. Modified the entry_points to adapt to these changes.
• Made the XmlRpc lib an optional component. There is more work to do here still.

[Ryan J Ollos]

(In [12090]) Refs #10206, #10251:

• Renamed timeline_hook.py to timeline.py, with the goal of making the source code layout match the standard for Trac plugins.
• Fixed an error introduced in [12089] due to a missing import.

As in [12088], if your trac.ini file specifies the individual components to enable, you'll have to make the following change:

worklog.timeline_hook.worklogtimelineaddon = enabled

->

worklog.timeline.worklogtimelineeventprovider = enabled

[Ryan J Ollos]

What the plugin needs now is someone to do more thorough testing than I've had time for. I would greatly appreciate the effort, and you can expect rapid follow-up and resolution of any issues that are found, within hours or a day at the most.

[Sam Halliday]

Thanks for this rjollos, but I never got to follow up when TH was down to say that I've now migrated to trac 1.0 :-)

[Ryan J Ollos]

I won't be doing any more work to support Trac < 1.0.