[Patch] TagsPlugin should handle versioning of wiki tags

[Daniel Kahn Gillmor]

Currently, tags on wiki pages as implemented by TagsPlugin are either present or not, with no record of who added the tag, when it was added, or if it was removed. Since the tagging table only contains a (tagspace,name,tag) tuple, there doesn't seem to be room in the current implementation for even storing the "author" of a tag on a certain wiki page, its history, or its destruction.

This seems contrary to the wikinature of the rest of trac, and certainly leaves the tickets (with their automatically revision-tracked keywords) looking much healthier. I'm not sure how one would make the transition for existing systems, but could the TagsPlugin be moved to a more wiki-ish model?

[Thomas Wagner]

I want this feature too. As a first step i can provide a patch which creates a new version of the wiki page if only the tags have changed. So you can see who added or removed tags and when it was done.

Perhaps this helps.

MfG WaTho

[lbruno@…]

Thanks MfG, I'll be using that patch. I'll also have to make some kind of timeline provider.

[Steffen Hoffmann]

I wonder, if we really need a new wiki page version.

1. Seems a bit overkill and too much overhead for a decent page and just some tags changing.
2. Deleting wiki versions will have no easy way to restore previous tags, and I'm not sure, if this would even be desired under all circumstances.

An alternative model would be to introduce versions directly to the default tag store and weave tag revisions into wiki page history like done for attachment changes in ticket change history for a long time. Attachments are not versioned yet, but I feel that the concept could work with versions too. Thought?

[Daniel Kahn Gillmor]

i like the idea of introducing versioning information (or just timestamps?) into the default tag store.

what if the tag store was expanded from:

• tagspace
• name
• tag

to:

• tagspace
• name
• tag
• added_by
• added_when
• removed_by
• removed_when

The "by"s refer to users, the "whens" are timestamps, and the "removed"s would initially be null.

you could make a view that matches the current default tag store just by selecting all rows where removed_when is null.

[Steffen Hoffmann]

Replying to dkgdkg:

i like the idea of introducing versioning information (or just timestamps?) into the default tag store.

what if the tag store was expanded ...

I envision a separate db table rather then extending the current table. Among other things it would allow to switch versioning on/off per realm (you likely don't need it for tickets). There should be only one author column, and repeated adding and removing of the same tag should not cause history overwrites. A db schema similar to ​`ticket_change` would allow for that with a minimal primary key extension and lower column count.

[anonymous]

Replying to hasienda:

I envision a separate db table rather then extending the current table. Among other things it would allow to switch versioning on/off per realm (you likely don't need it for tickets).

the tags for the ticketing system are all contained in the ticket keywords anyway, right? the tags table is only used for wiki pages.

There should be only one author column, and repeated adding and removing of the same tag should not cause history overwrites.

Ah right, my proposal was definitely flawed in that regard.

A db schema similar to ​`ticket_change` would allow for that with a minimal primary key extension and lower column count.

yep, i think this makes sense. if this was implemented, could we drop the tags table after an upgrade?

[Steffen Hoffmann]

Replying to anonymous:

Replying to hasienda:

I envision a separate db table rather then extending the current table. Among other things it would allow to switch versioning on/off per realm (you likely don't need it for tickets).

the tags for the ticketing system are all contained in the ticket keywords anyway, right? the tags table is only used for wiki pages.

No and no. Ticket keywords are configurable in [tags] section of trac.ini via the 'ticket_fields' option. 'keywords' is only its default value. Could be any combination of ticket fields (custom fields not supported yet, see #2804). But I don't feel that this is relevant here.

Wiki is not the only other tag-able realm today, there are blog and screenshots, not to speak about tomorrow.

There should be only one author column, and repeated adding and removing of the same tag should not cause history overwrites.

Ah right, my proposal was definitely flawed in that regard.

No problem. We discuss to find the best way.

A db schema similar to ​`ticket_change` would allow for that with a minimal primary key extension and lower column count.

yep, i think this makes sense. if this was implemented, could we drop the tags table after an upgrade?

No. As with ticket_change this will be only for the change records. It makes a great deal of flexibility to keep these information separately, only joining tables on demand. While it might be technically possible to get the same information from tag_change by examining tag history of a given resource, this would be far more expensive. Building a view would be another option, but for one this is not easy to do in a generic way (independent of db-backend's SQL flavor), and the Trac db schema is not highly normalized anyway.

[Daniel Kahn Gillmor]

OK, makes sense to me. thanks for the followup and clarification. and thank you for your work on this!

[Steffen Hoffmann]

You're welcome. I guess, that #11205 changed priority of this ticket significantly.

[Steffen Hoffmann]

preview at Trac 1.0 wiki change history view with tag revisions enabled

[Steffen Hoffmann]

I'll publish a patch for testing soon.

Here is a preview on current look-and-feel of tags version tracking for wiki pages backed by new tags_change db table. Tag change info in comments mimics keywords field history style in tickets. Links behind stars in version column lead to the wiki page version the tag change has been recorded for. Comments welcome.

[Steffen Hoffmann]

patch for an implementation of tag revisions (not only) for wiki pages

[Steffen Hoffmann]

Early adopters beware: This feature requires an additional db table. After installing a TagsPlugin version with this patch you'll need to upgrade your environment. Keep a backup for rolling back later on. Test reports are required to get it into next release tractags-0.7.

[Steffen Hoffmann]

In 13428:

TagsPlugin: Introduce tag version recording, requires db upgrade, refs #2429.

This will especially help recovering tags after SPAM tagging, refs #11205.
Therefore configurable behavior enables tracking of wiki page tag changes, but
excludes ticket keywords changes by default.

[Steffen Hoffmann]

In 13430:

TagsPlugin: Optimize tag change transactions, refs #2429.

Calculate effective differences instead of dumb delete-and-insert.
Additionally tag change recordes are removed now on resource destruction.

[Jun Omae]

I've tried it.

If wiki meta characters is uses in tag, those characters would explain as wiki in history view. I guess we should use ITemplateStreamFilter.filter_stream api.

1. Add 1.0 [{{{}}}] bar foo foobar plugin as tag
2. Replace the tags with abc**def bar baz foo

Also, the old version wiki view shows always the latest tags.

[Steffen Hoffmann]

Replying to jun66j5:

If wiki meta characters is uses in tag, those characters would explain as wiki in history view. I guess we should use ITemplateStreamFilter.filter_stream api.

I see. What you guess, is to escape these characters to keep them visible verbatim instead of interpreted as wiki syntax, right? I'll look into it.

Also, the old version wiki view shows always the latest tags.

I confess, that it even was intentional to not create real, revertible tag revisions at once. The change recording at least is complete, and more functionality could be added from there on as we see the need. But as the '*' suggests, re-tagging does not create a new wiki page revision. So when there is no such hard connection, is it really good to rollback younger tag changes for viewing older wiki page versions?

Of course there is always at least an order by change time. There are even more issues, like line truncating, so larger change sets are not displayed in full. Not that I would ignore it, I'm just unsure, if I can handle this all too soon.

[Jun Omae]

Replying to hasienda:

If wiki meta characters is uses in tag, those characters would explain as wiki in history view. I guess we should use ITemplateStreamFilter.filter_stream api.

I see. What you guess, is to escape these characters to keep them visible verbatim instead of interpreted as wiki syntax, right? I'll look into it.

Sorry, I've tried it and it's difficult to solve using filter_stream. But I found the workaround to replace wiki_to_oneline, ​33babfb.

Also, the old version wiki view shows always the latest tags.

Proposed changes can be found in ​f41d59c. The changes add tags_version parameter to query string in history view. The old tags for wiki page are searched by tags_version or version parameter.

​https://github.com/jun66j5/trac-tagsplugin/compare/ticket2429.

[Steffen Hoffmann]

The majority of the recent 7 proposed changes look just fine to me. I've commented them accordingly and invite you kindly to apply them - folded to 6 changes - to trunk here as well.

Obviously you don't see the need for my "tags change record composition" string. So it is not required for extra flexibility in some languages, isn't it? I thought giving a fixed positional order would be bad, but you do so by appending to the comment in your changeset ​33babfb150e8aef86dedca63a27c2d3471d10513.

Thank you very much for these contributions to refine the new feature. I'll update message catalogs after application of these patches to adapt to the i18n-related changes in there.

[Jun Omae]

Obviously you don't see the need for my "tags change record composition" string. So it is not required for extra flexibility in some languages, isn't it?

Indeed. You're right. I've updated ​ticket2429 branch based on your suggestions, and included some refactoring.

[Steffen Hoffmann]

It did isolated test of most of your changes by now, and I left my comments at GitHub. Would be great to have your contributions in trunk for the next release to come. Thanks for taking all the precious time to hack it.

[Steffen Hoffmann]

Replying to jun66j5:

Obviously you don't see the need for my "tags change record composition" string. So it is not required for extra flexibility in some languages, isn't it?

Indeed. You're right. I've updated ​ticket2429 branch based on your suggestions, and included some refactoring.

Just for clarity: I have finished the review with comment on ​afbcd4d in December, and I would appreciated, if you find the time to commit these latest changes to plugin's trunk branch yourself. If not, I would do it myself in fewer, but still isolated changesets for

• new RequestsProxy class,
• new private methods in wiki tag provider,
• method signature extension for setting time stamp explicitly and
• other fixes.

[Jun Omae]

Just for clarity: I have finished the review with comment on ​afbcd4d in December, and I would appreciated, if you find the time to commit these latest changes to plugin's trunk branch yourself. If not, I would do it myself in fewer, but still isolated changesets for

• new RequestsProxy class,
• new private methods in wiki tag provider,
• method signature extension for setting time stamp explicitly and
• other fixes.

Sorry for delay. Thanks for reviewing. I'll push proposed changes to trunk as 0.7 within a few days.

[Jun Omae]

In 13719:

TagsPlugin: small tweaks (refs #2429)

• iterate directly cursor rather than cursor.fetchall()
• remove redefined to_datetime in tractags.model
• prevent calling strip() twice in split_into_tags()

[Jun Omae]

In 13720:

TagsPlugin: prevent interpreting tagging comment as wiki in wiki history view (refs #2429)

[Jun Omae]

In 13721:

TagsPlugin: shows versioned tags for versioned wiki page (refs #2429)

[Jun Omae]

In 13723:

TagsPlugin: added tagn_ api which has tag_ and ngettext features, also tagn_ is one of extract keywords, now (refs #2429)

[Jun Omae]

In 13724:

TagsPlugin: store modified time of wiki page to tags_change.time when page is created and modified using request instance which is saved in threading.local (refs #2429)

[Jun Omae]

Sorry for delay. Thanks for reviewing. I'll push proposed changes to trunk as 0.7 within a few days.

The changes have been committed. Thanks.

[Steffen Hoffmann]

Replying to jun66j5:

Sorry for delay. Thanks for reviewing. I'll push proposed changes to trunk as 0.7 within a few days.

The changes have been committed. Thanks.

No, it's my pleasure to thank you for your dedication to make this feature complete. Now I feel confident to roll this new release. With this feature and substantial performance improvements this will become a remarkable milestone. I love to develop with such great co-contributors.

[Ryan J Ollos]

Replying to lbruno@…:

Thanks MfG, I'll be using that patch. I'll also have to make some kind of timeline provider.

I've created a new ticket for the timeline provider: #11661.

[Steffen Hoffmann]

In 13815:

TagsPlugin: Completing preparation for v0.7 release.

Availability of that code as stable, tagged release
closes #2429, #3359, #3610, #3624, #3677, #3754, #3864, #3947, #3983, #4078, #4277, #4503, #4799, #5523, #7787, #7857, #8638, #9057, #9058, #9059, #9060, #9061, #9062, #9063, #9149, #9210, #9521, #9630, #9636, #10032, #10416, #10636, #11096, #11147, #11152, #11274, #11302, #11658 and #11659.

Additionally there are some issues and enhancement requests showing progress,
but known to require more work to resolve them satisfactorily, specifically
refs #2804, #4200, #8747 and #9064.

Thanks to all contributors and followers, that enabled and encouraged a good
portion of this development work.