Changes between Initial Version and Version 1 of TestManagerForTracPluginGenericClass

Timestamp:

Jan 4, 2011, 5:25:48 PM (13 years ago)

Author:

Roberto Longobardi

Comment:

--

TestManagerForTracPluginGenericClass

@@ +1 @@
+= Test Manager for Trac - Generic Persistent Class Framework =
+
+The [wiki:TestManagerForTracPlugin Test Manager plugin] is comprised of four plugins, one of which is a '''Generic Persistent Class framework''', allowing plugin programmers to easily build and manage persistent objects in the Trac database. 
+
+Inheriting from the '''!AbstractVariableFieldsObject''' base class provided with this plugin is a matter of a few lines of code, and will provide your objects with:
+   * '''Persistence''', in the Trac database, 
+   * '''Custom Properties''', your Users will be able to specify in their trac.ini files. Also automatic Web user interface support is provided,
+   * '''Change History tracking''', recording User, timestamp and values before and after of every object's property change,
+   * '''Object lifecycle Listeners''', allowing your plugins or others to register for feedback on objects creation, deletion and modification,
+   * '''Custom Authorization''', allowing custom permissions to control any access to your objects,
+   * '''Integrated Search''', both via the Trac general Search box and programmatically, using pattern matching.
+
+[[BR]]
+The [wiki:TracGenericWorkflowPlugin Generic Workflow Engine] and the [wiki:TestManagerForTracPlugin Test Manager] plugins leverage this plugin for their data models.
+
+Works with both Trac 0.11 and 0.12.
+
+The basic building blocks of the framework are: 
+
+ * The '''!AbstractVariableFieldsObject''' base class, providing most object features. Inheriting from this class is a matter of a few lines of code, and will provide your objects with the features outlined above.
+
+ * The '''!IConcreteClassProvider''' interface, allowing your plugin to register your classes with the framework, participate in the framework's objects factory and provide custom security.
+
+ * The '''!GenericClassModelProvider''' component, providing the objects factory, managing concrete class providers and handling metadata about custom classes and their fields.
+
+ * The '''!GenericClassSystem''' component, providing the framework logic for displaying and updating custom properties in the Web user interface, and providing support for the Trac search box.
+
+ * The '''need_db_upgrade()''' and '''upgrade_db()''' utility methods, providing the ability for plugins to declaratively create and upgrade their database tables to match the custom classes provided.
+
+ * The '''IGenericObjectChangeListener''' inteface, an extension point interface for components that require notification when objects of any particular type are created, modified, or deleted.
+
+[[BR]]
+== !AbstractVariableFieldsObject - The data model ==
+
+The plugin provides a base python class, '''!AbstractVariableFieldsObject''', originally derived from the base Trac Ticket class, which brings a rich set of features to any custom class that should inherit from it.
+
+It represents a persistent object which fields are declaratively specified.
+
+Each subclass will be identified by a type name, the "realm", which is provided at object initialization time.
+This name must also match the name of the database table storing the corresponding objects, and is used as the base name for the custom fields table and the change tracking table (see below), if needed.
+
+Features:
+ * Support for custom fields, specified in the trac.ini file with the same syntax as for custom Ticket fields. Custom field values are kept in a "<schema>_custom" table.
+ 
+ * Keeping track of all changes to any field, into a separate "<schema>_change" table.
+
+ * A set of callbacks to allow for subclasses to control and perform actions pre and post any operation pertaining the object's lifecycle. Subclasses may give or deny permission for any operation to be performed.
+
+ * Registering listeners, via the IGenericObjectChangeListener interface, for object creation, modification and deletion. 
+
+ * Searching objects matching any set of valorized fields, (even non-key fields), applying the "dynamic record" pattern. See the method list_matching_objects.
+
+ * Very well commented code and full of debug messages.
+
+[[BR]]
+A set of special fields help subclasses to implement their logic:
+
+ * self.exists : always tells whether the object currently exists in the database.
+
+ * self.resource: points to a Resource, in the trac environment, corresponding to this object. This is used, for example, in the workflow implementation.
+
+ * self.fields: points to an array of dictionary objects describing name, label, type and other properties of all of this object's fields.
+
+ * self.metadata: points to a dictionary object describing further meta-data about this object.
+    
+[[BR]]
+Note: database tables for specific realms are supposed to already exist: this object does not create any tables.
+See below the !GenericClassModelProvider to see how to make the framework create the required tables declaratively.
+
+== How to create a custom persistent object ==
+
+This section will guide you with step-by-step instructions on how to create a custom object, including declaring the object's class, providing the fields metadata, and specifying the DB tables.
+
+Note: The following examples are taken from the [wiki:TracGenericWorkflowPlugin TracGenericWorkflow plugin]. Refer to the corresponding code if you want more context.
+
+The following is the '''basic set of imports''' you will need to create a custom object. The first one is needed to let your class inherit from the basic abstract class, the others to define your concrete model provider.
+
+{{{
+from tracgenericclass.model import AbstractVariableFieldsObject, IConcreteClassProvider, need_db_upgrade, upgrade_db
+}}}
+
+[[BR]]
+Then we must '''declare the concrete class'''. See the description below the code box.
+
+{{{
+class ResourceWorkflowState(AbstractVariableFieldsObject):
+    # Fields that have no default, and must not be modified directly by the user
+    protected_fields = ('id', 'res_realm', 'state')
+
+    def __init__(self, env, id=None, res_realm=None, state='new', db=None):
+        self.values = {}
+
+        self.values['id'] = id
+        self.values['res_realm'] = res_realm
+        self.values['state'] = state
+
+        key = self.build_key_object()
+    
+        AbstractVariableFieldsObject.__init__(self, env, 'resourceworkflowstate', key, db)
+
+    def get_key_prop_names(self):
+        return ['id', 'res_realm']
+        
+    def create_instance(self, key):
+        return ResourceWorkflowState(self.env, key['id'], key['res_realm'])
+}}}
+
+The first statement defines the concrete class as a subclass of AbstractVariableFieldsObject.
+
+The {{{protected_fields}}} field declares which fields must not be given a default at creation time. It is of little use, but it's still required.
+
+Then, the constructor. The only requirements on the constructor signature are that you must have room for the parameters {{{env}}} and {{{db}}}, which you'll need to pass to the superclass.
+The other parameters you see in this example are just specific to the workflow object.
+
+Talking about the constructor we must see the different uses you can make of it.
+
+In fact, you can:
+
+ * Create an empty object, and also try to fetch it from the database, if an object with a matching key is found.
+   To fetch an existing object from the database:
+   1. specify a key at contruction time: the object will be filled with all of the values form the database,
+   2. modify any other property via the {{{obj['fieldname'] = value}}} syntax, including custom fields. 
+      This syntax is the only one to keep track of the changes to any field.
+   3. call the save_changes() method on the object.[[BR]]
+ * Create a new object to be later stored in the database. In this case, the steps required are the following:
+   1. specify a key at contruction time,
+   2. set any other property via the {{{obj['fieldname'] = value}}} syntax, including custom fields,
+   3. call the insert() method on the object.[[BR]]
+ * Create an empty, template object, used for pattern matching search. To do this, do not specify a key at initialization time.[[BR]]
+
+In the body of the constructor, there a few important things you must do:
+ 1. Clear the {{{self.values}}} dictionary field, which holds all of the object's property values,
+ 2. Set the parameters received in the constructor into {{{self.values}}}. Use the syntax where you assign values directly to the {{{self.values}}} dictionary here, not the one like {{{self['fieldname'] = value}}},
+ 3. Build the object's key (by using the provided {{{self.build_key_object()}}} method, if you wish),
+ 4. Call the superclass constructor, passing as the third argument the unique type name that will identify objects of your custom class. Remember that this name must match the name of the database table providing persistence to your objects.
+
+Two more methods are required to make a functional subclass.
+
+The {{{get_key_prop_names()}}} method is vital in telling the framework which the key properties of your subclass are.
+As you can see, you should return an array of property names here. Also note that the {{{build_key_object()}}} method uses information returned from this method to build the object's key.
+
+The {{{create_instance()}}} is a factory method used to create an instance of this custom class, given a key dictionary object.
+
+[[BR]]
+'''This is it!!!''' Our new class is ready to be used, with all the features outlined at the top of this page.
+
+[[BR]][[BR]]
+Well, actually before being able to use our new class, we must:
+ * '''Define our concrete class provider''', implementing the '''!IConcreteClassProvider''' interface, and
+ * '''Declare the data model''' to be created into the Trac DB.
+
+But, as you will see, it is far from complex. See the description below the code box.
+
+{{{
+class GenericWorkflowModelProvider(Component):
+    """
+    This class provides the data model for the generic workflow plugin.
+    
+    The actual data model on the db is created starting from the
+    SCHEMA declaration below.
+    For each table, we specify whether to create also a '_custom' and
+    a '_change' table.
+    
+    This class also provides the specification of the available fields
+    for each class, being them standard fields and the custom fields
+    specified in the trac.ini file.
+    The custom field specification follows the same syntax as for
+    Tickets.
+    Currently, only 'text' type of custom fields are supported.
+    """
+
+    implements(IConcreteClassProvider, IEnvironmentSetupParticipant)
+
+    SCHEMA = {
+                'resourceworkflowstate':  
+                    {'table':
+                        Table('resourceworkflowstate', key = ('id', 'res_realm'))[
+                              Column('id'),
+                              Column('res_realm'),
+                              Column('state')],
+                     'has_custom': True,
+                     'has_change': True}
+            }
+
+    FIELDS = {
+                'resourceworkflowstate': [
+                    {'name': 'id', 'type': 'text', 'label': N_('ID')},
+                    {'name': 'res_realm', 'type': 'text', 'label': N_('Resource realm')},
+                    {'name': 'state', 'type': 'text', 'label': N_('Workflow state')}
+                ]
+            }
+            
+    METADATA = {
+                'resourceworkflowstate': {
+                        'label': "Workflow State", 
+                        'searchable': False,
+                        'has_custom': True,
+                        'has_change': True
+                    },
+                }
+
+            
+    # IConcreteClassProvider methods
+    def get_realms(self):
+            yield 'resourceworkflowstate'
+
+    def get_data_models(self):
+        return self.SCHEMA
+
+    def get_fields(self):
+        return self.FIELDS
+        
+    def get_metadata(self):
+        return self.METADATA
+        
+    def create_instance(self, realm, key=None):
+        obj = None
+        
+        if realm == 'resourceworkflowstate':
+            if key is not None:
+                obj = ResourceWorkflowState(self.env, key['id'], key['res_realm'])
+            else:
+                obj = ResourceWorkflowState(self.env)
+        
+        return obj
+
+    def check_permission(self, req, realm, key_str=None, operation='set', name=None, value=None):
+        pass
+
+    # IEnvironmentSetupParticipant methods
+    def environment_created(self):
+        self.upgrade_environment(self.env.get_db_cnx())
+
+    def environment_needs_upgrade(self, db):
+        return self._need_initialization(db)
+
+    def upgrade_environment(self, db):
+        # Create db
+        if self._need_initialization(db):
+            upgrade_db(self.env, self.SCHEMA, db)
+
+    def _need_initialization(self, db):
+        return need_db_upgrade(self.env, self.SCHEMA, db)
+}}}
+
+First of all, this should be a component, because it must implement Trac interfaces.
+The interfaces to implement are the following:
+ * IConcreteClassProvider, to provide custom classes to the framework,
+ * IEnvironmentSetupParticipant, to be queried by Trac about any needs regarding database creation or upgrade.
+
+Then here comes the declaration of the database schema, class fields and class metadata, to be later returned in the corresponding methods of the interface.
+
+Following is the '''!IConcreteClassProvider interface''' documentation, which is pretty explanatory about the format and meaning of the required data.
+
+{{{
+    def get_realms():
+        """
+        Return class realms provided by the component.
+
+        :rtype: `basestring` generator
+        """
+
+    def get_data_models():
+        """
+        Return database tables metadata to allow the framework to create the
+        db schema for the classes provided by the component.
+
+        :rtype: a dictionary, which keys are schema names and values 
+                are dictionaries with table metadata, as in the following example:
+                return {'sample_realm':
+                            {'table':
+                                Table('samplerealm', key = ('id', 'otherid'))[
+                                      Column('id'),
+                                      Column('otherid'),
+                                      Column('prop1'),
+                                      Column('prop2'),
+                                      Column('time', type='int64')],
+                             'has_custom': True,
+                             'has_change': True},
+                       }
+        """
+
+    def get_fields():
+        """
+        Return the standard fields for classes in all the realms 
+        provided by the component.
+
+        :rtype: a dictionary, which keys are realm names and values 
+                are arrays of fields metadata, as in the following example:
+                return {'sample_realm': [
+                            {'name': 'id', 'type': 'text', 'label': N_('ID')},
+                            {'name': 'otherid', 'type': 'text', 'label': N_('Other ID')},
+                            {'name': 'prop1', 'type': 'text', 'label': N_('Property 1')},
+                            {'name': 'prop2', 'type': 'text', 'label': N_('Property 2')},
+                            {'name': 'time', 'type': 'time', 'label': N_('Last Change')}
+                       }
+        """
+        
+    def get_metadata():
+        """
+        Return a set of metadata about the classes in all the realms 
+        provided by the component.
+
+        :rtype: a dictionary, which keys are realm names and values 
+                are dictionaries of properties.
+                
+                Available metadata properties are:
+                    label: A User-friendly name for the objects in this class.
+                    searchable: If present and equal to True indicates the class
+                                partecipates in the Trac search framework, and
+                                must implement the get_search_results() method.
+                    'has_custom': If present and equal to True indicates the class
+                                  supports custom fields.
+                    'has_change': If present and equal to True indicates the class
+                                  supports property change history.
+                    
+                See the following example:
+                return {'sample_realm': {
+                                'label': "Sample Realm", 
+                                'searchable': True
+                            }
+                       }
+        """
+
+    def create_instance(realm, props=None):
+        """
+        Return an instance of the specified realm, with the specified properties,
+        or an empty object if props is None.
+
+        :rtype: `AbstractVariableFieldsObject` sub-class instance
+        """
+
+    def check_permission(req, realm, key_str=None, operation='set', name=None, value=None):
+        """
+        Checks whether the logged in User has permission to perform
+        the specified operation on a resource of the specified realm and 
+        optionally with the specified key.
+        
+        Raise an exception if authorization is denied.
+        
+        Possible operations are:
+            'set': set a property with a value. 'name' and 'value' parameters are required.
+            'search': search for objects of this class.
+        
+        :param key_str: optional, the object's key, in the form of a string representing 
+                        a dictionary. To get a dictionary back from this string, use the 
+                        get_dictionary_from_string() function in the
+                        tracgenericclass.util package.
+        :param operation: optional, the operation to be performed on the object.
+        :param name: optional property name, valid for the 'set' operation type
+        :param value: optional property value, valid for the 'set' operation type
+        """
+}}}
+
+For what concerns the '''!IEnvironmentSetupParticipant interface''' implementation, it is really straightforward, in that the framework provides utility methods that, reading the schema declared in the above section:
+ * Determine whether a database upgrade is needed, checking the availability of the tables and the columns as declared. This is achieved through the utility method {{{need_db_upgrade()}}}.
+ * Perform the database upgrade if required. This is achieved through the utility method {{{upgrade_db()}}}. '''Note:''' only creating all the tables is currently supported, not altering existing tables (for example due to your plugin database schema changing between versions).
+
+[[BR]]
+== !IGenericObjectChangeListener - Registering listeners to objects lifecycle events ==
+
+Every class inheriting from !AbstractVariableFieldsObject supports a listener interface for components interested in objects' lifecycle events.
+
+To register a listener for any particular class type, i.e. "realm", your component must implement the {{{!IGenericObjectChangeListener}}} from the {{{tracgenericclass.api}}} package.
+
+Following is the documentation from the interface itself.
+
+{{{
+    def object_created(g_object):
+        """Called when an object is created."""
+
+    def object_changed(g_object, comment, author, old_values):
+        """Called when an object is modified.
+        
+        `old_values` is a dictionary containing the previous values of the
+        fields that have changed.
+        """
+
+    def object_deleted(g_object):
+        """Called when an object is deleted."""
+}}}
+
+
+
+