Allow overriding of actual ticket times with user times

[fabian.freyer@…]

At the moment the precedence is

1. if useActuals is set, use the actual time
2. else if there is a cached date in the db, use that
3. else if there is a user-supplied field set, use it
4. otherwise compute the dates

I would like the option to change the order of precedence for these policies. Especially, it would be great to be able to override the actual ticket time with a user field, e.g. if a ticket was re-opened or closed long after its real completion, so that it doesn't mess up the gantt chart.

In my case, I would want to use the following order:

1. Use the user-supplied field, if it is set
2. Else if useActuals is set and the ticket was closed, use the actual date,
3. Else if a cached date field exists, use it
4. Else compute the dates.

[Chris Nelson]

An interesting idea. What would you suggest as a means to configure or select this?

Is a re-opened ticket your primary use case? Would it be enough for the scheduler to favor the user-supplied time for a ticket which had been closed but it no longer closed?

[fabian.freyer@…]

As a means to configure this, I would suggest a ListOption or OrderedExtensionsOption to reorder the time override policies. Instead of the if/elif/elif/else block, I would then iterate over the option values and override the last value if the current policy applies. The default value should then be equivalent to the current order of precedence.

I'll try to whip up a patch later. Unfortunately, I don't have a nice testing environment for trac development yet, so I'll try to set that up first :)

A reopened ticket is not my primary use case, but a main one. My secondary use case is a later adjustment of actual completion dates.

[Chris Nelson]

Replying to fabian.freyer@…:

As a means to configure this, I would suggest a ListOption or OrderedExtensionsOption to reorder the time override policies. Instead of the if/elif/elif/else block, I would then iterate over the option values and override the last value if the current policy applies. The default value should then be equivalent to the current order of precedence. ...

I was thinking of something like that. So, sfPrecedence (start/finish precedence) would default to:

[ actual, db, ticket, computed ]

but you might set it to

[ ticket, actual, db, computed ]

Where "ticket" is what the user specified in the ticket and "db" is the "cached" value.

And this would be used in a

for dateType in sfPrecedence:
   if dateType data available, use it and break out of loop

(I wonder if "computed" is always a fallback? Does it need to be configurable or can you just order the other three and if nothing is found, compute?)

I assume you'd want to be able to specify this in trac.ini but also per-chart (e.g., [[TracJSGanntChart(...,sfPrecedence=ticket|actual|db|computed,,,,)]])?

[fabian.freyer@…]

Replying to ChrisNelson:

I was thinking of something like that. So, sfPrecedence (start/finish precedence) would default to:

[ actual, db, ticket, computed ]

but you might set it to

[ ticket, actual, db, computed ]

Where "ticket" is what the user specified in the ticket and "db" is the "cached" value.

yes, that sounds good. This could also deprecate useActual as you could just remove actual from sfPrecedence, i.e. the default (useActual=0) would be

sfPrecedence=[db,ticket,computed]

(I wonder if "computed" is always a fallback? Does it need to be configurable or can you just order the other three and if nothing is found, compute?)

"computed" always returns a start/finish time, right? So, as long as "computed" in sfPrecedence, no fallback would be needed. Therefore, I would suggest something like

if "computed" not in sfPrecedence:
    sfPrecedence.append("computed") # use computed as a fallback if all else fails

class DateNotSet(Exception):
    pass

for policy in sfPrecedence:
    try:
        (start,finish) = sfPolicies[policy]()
        break
    catch(DateNotSet):
        pass

I assume you'd want to be able to specify this in trac.ini but also per-chart (e.g., [[TracJSGanntChart(...,sfPrecedence=ticket|actual|db|computed,,,,)]])?

I hadn't thought of that, but it sounds like a useful feature. This could also make things like "planned" and "actual" gantt charts possible.

[anonymous]

actually, the try/catch block should be

    try:
        (start,finish) = sfPolicies[policy]()
    catch(DateNotSet):
        pass
    else:
        break

[fabian.freyer@…]

I have created this patch, which should add this option. However the feature of being able to set sfPrecedence per-chart is not implemented yet.

I was able to test this with the "actual" date policy. However, I don't have a test environment ready. Could you confirm this works as expected?

tracjsgantt/tracpm.py

@@ -20 +20 @@
 except ImportError:
     from trac.util.datefmt import to_timestamp as to_utimestamp
 
-from trac.config import IntOption, Option, ExtensionOption
+from trac.config import IntOption, Option, ExtensionOption, ListOption
 from trac.core import implements, Component, TracError, Interface, ExtensionPoint
 from trac.env import IEnvironmentSetupParticipant
 from trac.db import DatabaseManager
@@ -185 +185 @@
            """List of statuses for goal-type tickets that are active""")
     Option(cfgSection, 'useActuals', '0',
            """Use actual start, finish date for tickets""")
-
     scheduler = ExtensionOption(cfgSection, 'scheduler',
                                 ITaskScheduler, 'ResourceScheduler')
+    sfPrecedence = ListOption(cfgSection, 'sfPrecedence',
+        default=['db', 'ticket', 'computed'], sep=",",
+        doc="""
+        Order of precedence of date policies.
+        This is a comma-separated list, first values will be prioritized.
+        Valid policies are: actual, db, ticket, computed.
+        """)
 
     def __init__(self):
         self.env.log.info('Initializing TracPM')
@@ -1742 +1748 @@
             SF_PROJECT = 5
             SF_DEFAULT = 6
 
-            # If we haven't scheduled this yet, do it now.
-            if t.get('_calc_' + fromField) == None:
-                self._logSch('Scheduling %s' % t['id'])
+            class DateNotSet(Exception):
+                pass
 
-                # Use actual dates, if requested.
-                if t.get('_actual_' + fromField) and options.get('useActuals'):
-                    taskFrom = [ to_datetime(t['_actual_' + fromField]),
+            def date_actual(task, fields, direction):
+                """ Use actual dates, if requested."""
+                if task.get('_actual_' + fields[direction]):
+                    task_date = [ to_datetime(task['_actual_'
+                                 + fields[direction]]),
                                  SF_ACTUAL ]
                     self._logSch('Using actual %s:%s' %
-                                 (fromField, taskFrom[0]))
-                # If there is a precomputed date in the database,
-                # use it unless we're forcing a schedule calculation.
-                elif t.get('_sched_' + fromField) and not options.get('force'):
-                    taskFrom = [ to_datetime(t['_sched_' + fromField]),
+                                 (fields[direction], task_date[0]))
+                    return task_date
+                else:
+                    raise DateNotSet()
+
+            def date_cached(task, fields, direction):
+                """
+                If there is a precomputed date in the database,
+                use it unless we're forcing a schedule calculation.
+                """
+                if task.get('_sched_' + fields[direction]) \
+                  and not options.get('force'):
+                    task_date = [ to_datetime(task['_sched_'
+                                 + fields[direction]]),
                                  SF_SCHEDULE ]
-                    self._logSch('Using db %s: %s' % (fromField, taskFrom[0]))
-                # If there is a user-supplied date set, use it
-                elif self.pm.isSet(t, fromField):
+                    self._logSch('Using db %s: %s' %
+                        (fields[direction], task_date[0]))
+                    return task_date
+                else:
+                    raise DateNotSet()
+
+            def date_user(task, fields, direction):
+                """ If there is a user-supplied date set, use it """
+                if self.pm.isSet(task, fields[direction]):
                     # Don't adjust for work week; use the explicit date.
-                    taskFrom = self.pm.parseTaskDate(t, fromField)
-                    taskFrom = [taskFrom, SF_TASK]
+                    task_date = self.pm.parseTaskDate(task, fields[direction])
+                    task_date = [task_date, SF_TASK]
                     self._logSch('Using explicit %s: %s' %
-                                 (fromField, taskFrom[0]))
-                # Otherwise, compute from date from dependencies.
+                                 (fields[direction], task_date[0]))
+                    return task_date
                 else:
-                    taskFrom = dependentLimit(t, ancestorLimit(t))
+                    raise DateNotSet()
+
+            def date_computed(task, fields, direction):
+                """ compute date from dependencies """
+                if direction == "from":
+                    taskFrom = dependentLimit(task, ancestorLimit(task))
 
                     # The date derived from dependencies is *not* a
                     # fixed (user-specified) date.
                     if taskFrom != None:
                         self._logSch('Got %s from dependencies: %s' %
-                                     (fromField, taskFrom[0]))
+                                     (fields["from"], taskFrom[0]))
                         taskFrom[1] = SF_DEPENDENCIES
                     # If dependencies don't give a date, use date from
                     # project.  Default to today if none given.
                     else:
                         # Get user-supplied date for schedule.
-                        taskFrom = self.pm.parseDbDate(options.get(fromField))
+                        taskFrom = self.pm.parseDbDate(
+                            options.get(fields["from"]))
                         # If none, use midnight today
                         if taskFrom == None:
                             taskFrom = datetime.today().replace(hour=0,
@@ -1788 +1816 @@
                                                               microsecond=0,
                                                               tzinfo=localtz)
                             self._logSch('Defaulting %s: %s' %
-                                         (fromField, taskFrom))
+                                         (fields["from"], taskFrom))
                             taskFrom = [taskFrom, SF_DEFAULT]
                         else:
                             self._logSch('Using project %s: %s' %
-                                         (fromField, taskFrom))
+                                         (fields["from"], taskFrom))
                             taskFrom = [taskFrom, SF_PROJECT]
+                    return taskFrom
+                else:
+                    hours = self.pm.workHours(task)
+                    taskTo = task['_calc_' + fields["from"]][0] + \
+                        _calendarOffset(task,
+                                        dir * hours,
+                                        task['_calc_' + fields["from"]][0])
+                    taskTo = [taskTo, task['_calc_' + fields["from"]][1]]
+                    self._logSch('Computed %s from %s, work: %s' %
+                                 (fields["to"], fields["from"], taskTo[0]))
+                    return taskTo
+
+            sfPolicies = {
+                'actual': date_actual,
+                'db': date_cached,
+                'ticket': date_user,
+                'computed': date_computed
+            }
 
+            # If we haven't scheduled this yet, do it now.
+            if t.get('_calc_' + fromField) == None:
+                self._logSch('Scheduling %s' % t['id'])
+                for policy in self.pm.sfPrecedence:
+                    try:
+                        if policy not in sfPolicies:
+                            raise TracError("Invalid policy %s. Check sfPolicies setting." % policy)
+                        taskFrom = sfPolicies[policy](t,
+                            {"from": fromField, "to": toField},
+                            "from")
+                    except DateNotSet:
+                        pass
+                    else:
+                        break
 
                 # Check resource availability.
                 #
@@ -1832 +1892 @@
             # function.  Specifically, when the toField is set in the
             # task.
             if t.get('_calc_' + toField) == None:
-                # Use actual dates, if requested.
-                if t.get('_actual_' + toField) and options.get('useActuals'):
-                    taskTo = [ to_datetime(t['_actual_' + toField]), SF_ACTUAL ]
-                    self._logSch('Using actual %s: %s' %
-                                 (toField, taskTo[0]))
-                # If there is a precomputed date in the database,
-                # use it unless we're forcing a schedule calculation.
-                elif t.get('_sched_' + toField) and not options.get('force'):
-                    taskTo = [ to_datetime(t['_sched_' + toField]),
-                               SF_SCHEDULE ]
-                    self._logSch('Using db %s: %s' % (toField, taskTo[0]))
-                # If there is a user-supplied date set, use it
-                elif self.pm.isSet(t, toField):
-                    taskTo = self.pm.parseTaskDate(t, toField)
-                    taskTo = [taskTo, SF_TASK]
-                    self._logSch('Using explicit %s: %s' %
-                                 (toField, taskTo[0]))
-                # Otherwise, the to date is based on the from date and
-                # the work to be done.
-                else:
-                    hours = self.pm.workHours(t)
-                    taskTo = t['_calc_' + fromField][0] + \
-                        _calendarOffset(t,
-                                        dir * hours,
-                                        t['_calc_' + fromField][0])
-                    taskTo = [taskTo, t['_calc_' + fromField][1]]
-                    self._logSch('Computed %s from %s, work: %s' %
-                                 (toField, fromField, taskTo[0]))
-
+                for policy in self.pm.sfPrecedence:
+                    try:
+                        if policy not in sfPolicies:
+                            raise TracError("Invalid policy %s. Check sfPolicies setting." % policy)
+                        taskTo = sfPolicies[policy](t,
+                            {"from": fromField, "to": toField},
+                            "to")
+                    except DateNotSet:
+                        pass
+                    else:
+                        break
                 t['_calc_' + toField] = taskTo
 
             # Adjust dates based on precedence