Synchronize User Account data between multiple Trac projects

Warning: This plugin is at best an early beta. So take special care using it, especially in production environments!

Description

This plugin can be used to synchronize user account data between multiple projects within the same TRAC_PARENT. It reads the account information from all the separate environments, merges them, and then updates all environments accordingly. As it currently still is in early beta, a dryrun mode is enabled by default - so no changes will be written to the database. Instead, the updates for the affected environments will be stored in <tracenv>.sql files to investigate.

If you intend to use this plugin in its current state, you are strongly encouraged to:

1. first test it in a non-productive environment, best is a copy of the intended productive environment to be used on later
2. first let it run in drymode, and investigate the resulting *.sql files carefully (see below)
3. only if everything seems to be fine, and still being in your test environment, and after making a copy of the databases, disable drymode and verify the results
4. only if still everything seems to be OK, repeat these steps in your productive environment - at your own risk

If you are still eager to try it: Great - I love to receive your feedback!

Limitations

There are the following limitations to this plugin:

1. it is not widely tested yet
2. it is not yet "fool-proof" (if it ever will be). This especially means, not all possible exceptions may be handled yet
3. it works only for environments, which meet the following conditions:
   1. share the same TRAC_PARENT_DIR
   2. use the AccountManagerPlugin
   3. store the user passwords in a .htpasswd file
   4. share the same .htpasswd password file

Bugs/Feature Requests

Existing bugs and feature requests for TracUserSyncPlugin are here.

If you have any issues, create a new ticket.

defect	0 / 1
task	0 / 1

Download

Download the zipped source from here.

Source

You can check out TracUserSyncPlugin from here using Subversion, or browse the source​ with Trac.

Versions:

• 0.1: Basic synchronization
• current trunk: adds purge functionality

Installation

The easiest way to install this plugin is:

easy_install https://trac-hacks.org/svn/tracusersyncplugin/0.11/trunk

While you may want to replace "trunk" by one of the branches. "trunk" is usually the "code in development", while the branches should reflect something "stable".

You can also checkout the code from the repository, or download and unpack the zipped source (see above) - and then run either easy_install or python setup.py from where the setup.py file resides.

Then you need to activate the plugin, by adding the following in the components section of your trac.ini file:

[components]
tracusersync.* = enabled

Configuration

For testing purposes, the default settings should be fine. However, there are some additional settings in the [user_sync] section of your trac.ini file:

[user_sync]
dryrun = true
merge_conflicts = skip
sql_file_path =
sync_fields = email,name
users_keep =
exclude_envs =

The easiest way to modify these settings is using the IniAdminPlugin, where you always have some helpful information displayed next to the options. Here are some brief explanations:

dryrun

This enables the "test mode", in which no changes will be done to your environments including the databases. Instead, changes which would be written to the database will be stored in *.sql files, one for each environment. This is a boolean setting - so the only valid values here are "true" and "false".

merge_conflicts

What should be done if records from two (or more) environments conflict. Possible values are "skip" (do not update this user anywhere) and "newer" (use the record from the environment the user was last active in, which is not necessarily the one with the most recent data). Default is "skip", to be on the safe side.

What makes a conflict? Say user Tom registers in environment A, and sets his name to "Tom Sawyer" and his email to "tom@…". His password is stored to the shared .htpasswd file, so he can immediately login to environment B without registering again. He does so - but edits his record here, using the name "Tommy". If the field name is contained in the sync_fields list (see below), this would cause a conflict since the two names don't match - which means, the email won't be synchronized either. The same applies the other way round: if a different email was specified in one environment, the entire record would be considered conflicting.

sql_file_path

The path where the *.sql files shall be stored into. If not set, they will be written into the log sub directory of the environment the plugin was invoked from.

sync_fields

Which fields of the user records should be considered for synchronization. By default, this is set to name,email, the two basic fields. If you use the UserManagerPlugin, you may want to add some more fields. Note that for now a single conflict on any of the fields will exclude a user record from being merged, so the more fields added, the higher the chances are for conflicts.

There are two more fields considered here, even if not mentioned (and you should never introduce them into that list): the information used by the email verification. This means, if you enabled email verification in the AccountManagerPlugin, we will try to take care for that as well. So if a user verified for one environment, we try to do this for the other environments as well.

users_keep

This option only affects purging: here you can define users which shall always be excluded from purging. The plugin will take care for your permission groups (e.g. anonymous, authenticated) automatically, so you don't need to add those (if your check - see below - shows the plugin missed one, you might add it here, though).

exclude_envs

A comma separated list of environments to exclude from our actions by default. The only effect here is that the corresponding check boxes in the web_ui will not be checked then.

Usage

Settings

You will find the interface on the Admin page in the Accounts section as User Sync (see also screenshot). After invoking this page, you can select the environments to synchronize. The list includes all Trac environments sharing the TRAC_PARENT_DIR with the environment you are currently in - though it did not yet check what password store they are using or even whether they share the same password file (this may be added later). If your selection includes an environment with conflicting data, it will be excluded later. All found environments are pre-selected except those mentioned in the exclude_envs setting, so you might want to (un)check one/some or leave it as is.

The second "field set" shows you the password file used by the current environment, and what users have been stored there. This is just for your information.

The third section lets you select the actions to perform. Synchronize is already pre-selected here, since this is the main task this plugin is for, and also is what you probably want to do. This action means: Make sure all included environments get the account data users have entered in one of the environments for fields contained in the sync_fields setting. The second check box, labeled Purge, will cause the plugin to remove all users from all environments, except those contained in the password file or mentioned in the users_keep setting. Of course, it would not touch your permission groups (anonymous, authenticated and whatever you may have added), these are evaluated automatically.

WARNING: You should always run in dryrun mode first and check the resulting *.sql files carefully. Every Trac setup has its specifics, and I only tested the script in my own setup. So as long as there's no feedback from other users, I'd consider it not-yet-safe!

Results

Now that you've made your selections, push the Perform actions button. When the page is reloaded, it will inform you about the results:

A green box on top (see below screenshot) will give you short information about successful operation. If you got a red box instead, the operation failed completely.

At the end of the page, a new "field set" called Log messages appears, giving you more detailed information about the process. This includes successful steps as well as "minor failures" (such as environments excluded for reasons like missing privileges or conflicting settings). Read this carefully to get an overview.

Next, you should also investigate the resulting *.sql files. Some hints:

• Check the NOT IN () list of the DELETE statements to make sure it contains all the users of your password file (they should be in here) as well as all your permission groups (should be in as well - but the plugin may have failed to set them all up correctly - remember, it's not yet thoroughly tested) and users_keep
• Check the INSERT statements for each environment (e.g. by turning them into SELECT statements and run those against the database - they should return no data then)
• Check the UPDATE statements in the same way, if there are any - in this case, the corresponding SELECT must return a record, but not necessarily with the same data (hence the update)

Hot run

You may disable the dryrun mode once you finished the checks above. Well, you may even disable it immediately - but that's currently not recommended. Once you are sure everything works fine, you can even disable the writing of the *.sql files by setting the sql_file_path = none in your trac.ini.

Example

This is what it could look like after a successful synchronization:

Planned features

Here are some things for future consideration:

1. Only list "compatible" environments, ie those sharing the same password file, and where the current user has the required privileges
2. Make the plugin more "fool-proof" and reliable (I need your feedback for that)
3. Identify and remove "dummy users", ie spammers whose spam you already removed: registered users with no ownership/authorship of any ticket, wiki article, component, whatsoever - and a last_visit xx days in the past
4. Introduce some "Listeners" to notice registrations/changes in environments and automatically propagate them to the other environments, so you only need to run the synchronization manually once - which is after installation.
5. Reduce the amount of fields that conflicts are checked for: this would result in allowing a higher amount of overlap between attributes of users and reduce the amount of conflicts.

Recent Changes

7733 by izzy on 2010-02-26 07:03:06

fixed a typo and added some more exception handling (usersync crashed under some strange conditions)

6987 by izzy on 2009-11-01 18:15:44

! wrong defaults for excludes

6986 by izzy on 2009-11-01 18:14:38

+ added option to exclude environments form sync by default
! selection of environments was ignored

(more)

Author/Contributors

Author: izzy
Maintainer: izzy
Contributors: