Mass tag replacement functionality

[anonymous]

It should be possible to rename all instances of an existing tag with one admin-command.

Patch in #9059 adds such functionality using an admin panel.

[Steffen Hoffmann]

While I'm fine with introducing such additional functionality and making it accessible via a dedicated 'tags' admin page, I recommend to clearly differentiate between such administrative tasks and changing individual tags on a per-resource base.

IMHO something like mass rename is certainly at a different level, so permission shouldn't be granted too broadly to such a powerful action. Therefore [10627] introduced a 3rd permission TAGS_ADMIN in advance.

[Steffen Hoffmann]

Thinking it through for a while now it seems like 'rename' is the wrong description for what we do here.

Since a tag is just the label - no object behind apart from an optional tag wiki page and the tag's parent resource, the change should better be described as replacement.

[Steffen Hoffmann]

Consider the following hypothetical TagCloud:

anything mything something thatthing thing this thisthing thsithing yourthing

where we have three times the tag 'thisthing', but 2 are wrong, caused by bad user input: 'this thing' (been broken into two tags) and 'thsithing' as well as the similar 'thatthing'. It would take 4 runs of the upcoming replace functionality to move to the desired sane state. Similar situations could arise, where one would like to reduce diversity by merging synonyms or words with almost equal meaning into a more stringent set of tags.

Question: Would it be good to elaborate the replace feature a little more into a consolidate/replace, where we could select even multiple tags to consolidate into one tag within a single transaction, wouldn't it? Still simple replacements aka 'renaming' could be done, since consolidation is a super-set of this operation, right?

I'd like to hear your opinion before pushing this out. Thanks in advance for taking your time.

[Itamar Oren]

This consolidation you described sounds pretty good!

Few things to note:

• The simple one-to-one 'renaming' would be a subset of consolidation only if you allow the "target tag name" to be a free-text-field, since you might want to change all instances of some tag into something that doesn't yet exist (e.g. I used to tag things in singular form ('thing') and I want to change it all to plural form ('things')).
• When you add to the interface the ability to check (or select) tags from the existing cloud, you might overload the UI with a very big list. Several options to deal with this:
  1. Don't list all tags. Give an input-box with intellisense functionality based on existing tags. (I imagine here the "To" field when I compose an email in Gmail, where I can start typing a name and I get a dropdown of relevant suggestions, and I select an option using tab or mouse-click and can input another name for another recipient)
  2. Don't list all tags. Give a free-text box with a "Show cloud" button next to it (like "Browse" for file selection) that opens a dialog with the full tag cloud that returns the user selection to the free-text box.
  3. Best option in my opinion: Combine both! :-)

Either way - all this is also relevant to the tags-input-box in wiki-edit!

[Steffen Hoffmann]

Replying to itamarost:

This consolidation you described sounds pretty good!

Good, so it's more than only a wild idea in my head.

Few things to note:

• The simple one-to-one 'renaming' would be a subset of consolidation only if you allow the "target tag name" to be a free-text-field, since you might want to change all instances of some tag into something that doesn't yet exist ...

Yes, this is how I thought about it too.

• When you add to the interface the ability to check (or select) tags from the existing cloud, you might overload the UI with a very big list. Several options to deal with this:
  1. Don't list all tags. Give an input-box with intellisense functionality based on existing tags. ![...]

This has already been invented is know as KeywordSuggestPlugin. I like it for every Trac application I setup, and it's designed to work in concert with TagsPlugin.

1. Don't list all tags. Give a free-text box with a "Show cloud" button next to it (like "Browse" for file selection) that opens a dialog with the full tag cloud that returns the user selection to the free-text box.

If I hear 'selection', I think of the users admin page of AccountManagerPlugin. This is designed to cope well even with hundreds of list entries, a table view with checkbox in front of each line. If I hear 'tag cloud' I can't see more than a single click per call, and this seem like a tedious selection procedure, not much favorable over a dropdown.

Actually I started with a select list (see this example​), but still have to find out about multi-select.

1. Best option in my opinion: Combine both! :-)

Hm, will have to think about multiple input option. Must retain a clear, intuitive interface as far as I'm concerned with usability.

Either way - all this is also relevant to the tags-input-box in wiki-edit!

Well, you're right, but KeywordSuggestPlugin works well as outlined above, so I wouldn't bother reinventing a similar thing, only re-use it.

[Steffen Hoffmann]

screenshot of tags admin page in development for tags-0.7 on Trac 0.13dev

[Steffen Hoffmann]

(In [10637]) TagsPlugin: Refactor author retrieval from tag change request, refs #9059 and #9061.

Code threw an AttributeError: username exception when attempting to change tags on tickets. Couldn't find another occurance of req.username in any other plugin nor in Trac core itself.

Turns out this flaw has been specific to the 0.6 release introduced by the first development code in changeset [2953] back in 2008 - wow. I stumbled upon this when attempting to implement tag mass replacement - a lucky catch. Lucky again for finding get_reporter_id, that has been avaiable since earliest days - 2004, and it works very well.

[Steffen Hoffmann]

screenshot of updated tags admin page in development for tags-0.7 on Trac 0.13dev - now with multiple select and help text

[Steffen Hoffmann]

The updated screenshot​ shows the current development state. Comments welcome.

[Itamar Oren]

Replying to hasienda:

The updated screenshot​ shows the current development state. Comments welcome.

Very nice.

Do you know of tag providers for which the removal would fail, as mentioned in the disclaimer you wrote?

[osimons]

Reading through the replace code, I think it should work for the tags provider included in FullBlogPlugin. Untested though.

[Steffen Hoffmann]

Replying to itamarost:

Replying to hasienda:

The updated screenshot​ shows the current development state. Comments welcome.

Very nice.

Thank you.

Do you know of tag providers for which the removal would fail, as mentioned in the disclaimer you wrote?

Yes, I didn't make that one up. I've tested with

[tags]
ticket_fields = keywords, component, reporter, cc, owner, version, milestone

and - as expected - you can't remove any tag derived from fields other than keywords, right? So I put in the hint to ticket_fields option as shortcut for anyone else. Still no idea, if any other tag provider out there is similar to the one for the ticket tag realm.

[Steffen Hoffmann]

Replying to osimons:

Reading through the replace code, I think it should work for the tags provider included in FullBlogPlugin. Untested though.

Would be good to know, if you can afford to test the reworked code after I've commited it (within the next few days).

[Steffen Hoffmann]

(In [10638]) TagsPlugin: Allow optional comment on tag changes, refs #9061.

This is an API change.

It looks tiny, still any Component implementing ITagProvider should be updated to fully support new capabilities. This is sensible at least, if its realm knows about tag changes like Trac tickets does, where we use the keywords field, that has changes recorded like other ticket fields in Trac db table ticket_change.

[Steffen Hoffmann]

(In [10639]) TagsPlugin: Add tag mass replacement functionality, refs #9058, #9059 and #9061.

This is a modified version of code originally contributed by Itamar Ostricher. I felt like improving usability by replacing the input field for old tag(s) with a select box containing all currently defined tags, where even multiple selection and tag mass deletion is possible now.

Thank you for your truely inspiring work, Itamar.

[Steffen Hoffmann]

Despite of testing with Trac 0.11 and 0.13dev for some time now, this new feature may not be 100 % bug free.

Anyway, I've pushed it out to hear your experience. Tag management is a new dimension for the TagsPlugin. Test, enjoy.

Hint: There is a tiny, hidden feature. If you type an existing tag into New Tag: input field and hit <Change>, you'll get the tag selected in the Current Tag: select field above. May help with quick checks for existence in overly long tag lists, or just confuse you on testing with arbitrary, random input. :-)

[osimons]

My FullBlogPlugin tags provider worked without any issues, testing with current trunk. Tags (= post categories) was updated nicely via admin.

[Steffen Hoffmann]

Replying to osimons:

My FullBlogPlugin tags provider worked without any issues, ![...]

Hey, thanks for instant testing. Very encouraging indeed. You wanna push me to get a new release out, right? ;-)

[Steffen Hoffmann]

Before finally putting this into the release queue as it is now, I want to raise another rant on my own work:

I found it convenient to include mass tag deletion during the development, mostly because it was easy to implement as a side-effect. But I didn't like to rename the admin page to 'Replace/Delete' or similar, nor would I like to advertise that functionality too loudly for one reason: I feel tagging is a per-resource action, adding information that would have been not so obvious to retrieve by other means.

Something like mass tagging or tag deletion is not a great idea in general, as it defeats this 'add non-obvious information' idea (spell: hand-picked resources) and should be reserved for uncommon situations, like SPAM tagging (if this is even an issue, never seen that happen by now).

What do you think?

[osimons]

The 'replace' functionality is no doubt useful enough to help manage large amounts of user-submitted content. Isn't "replacing with nothing" (aka 'delete') just a variation of this?

Mass-adding on the other hand would however be a very different beast to handle, and you'd need to somehow make a selection of resources based on non-tags criteria to be really useful. Which is quite a difficult coding task seeing no two resources are alike or constructed / queried the same way. This is better left to each subdomain to implement, for instance mass-adding of keywords for tickets using the BatchModifyPlugin.

The only 'add' that would make sense in a Tags context is to 'replace' one tag with two or more other tags. Typically if you would want to tag at a finer grain than currently used, splitting old content into several new dimensions.

It is all 'Replace' as I see it.

[Steffen Hoffmann]

Replying to osimons:

![...] The only 'add' that would make sense in a Tags context is to 'replace' one tag with two or more other tags. Typically if you would want to tag at a finer grain than currently used, splitting old content into several new dimensions.

Interesting, that you mention this without me mentioning this before. But I thought about this, and well along the same lines. So far we have many-to-one and one-to-one replacements. One-to-many is not implemented now, but at least two of us would rate this as valuable. So may it be, I'll try.

It is all 'Replace' as I see it.

Just as I see it too. Thanks for your confirmation.

[Ryan J Ollos]

I think I've found a defect with this feature. I'm running r10800.

If you have a comma separated list of tags in the keywords field of a ticket, e.g. tag1, tag2 and you replace tag2 -> tag3 via the WebAdmin panel, the resulting keywords field for the ticket will be tag1 tag3. That is, the comma is removed.

Here are some examples from my Trac instance:

Renamed AcousticCalibration -> acoustic-calibration:

• Keywords AcousticCalibration string-phantom added; AcousticCalibration, StringPhantom removed

• Keywords changed from TestAndMeasurement, CheckSource, T&M, AcousticCalibration to CheckSource T&M TestAndMeasurement acoustic-calibration.

[Steffen Hoffmann]

I don't remember: Is there a config option for the keyword separator, or is it just free-style? In the latter case we would need to detect/guess (ugly), in the former just take the setting. You know about it?

[Ryan J Ollos]

I think it is just free-style. On my Trac instance, I think the separator is added by the KeywordSuggestPlugin, for which their is a TracIni option.

[Steffen Hoffmann]

This option came instantly into my mind as well.

While I'm thinking about this, it seems like Itamar is already suggesting a fix in #9692. It would be better to have this option in the [tags] section, but don't know, how to do such an operation in a backwards-compatible way. Even re-using is not as easy as it looks in first sight, because he simple config.get approach would imply a dependency on KeywordSuggestPlugin.

[Steffen Hoffmann]

This will get finally fixed unexpectedly by inclusion of KeywordSuggestPlugin.

There's #9692 for tag list normalization and configurable tag separators. So this would be more appropriate for following up on this remaining issue.

[Steffen Hoffmann]

In 13304:

TagsPlugin: Add realm filter to tag administration, refs #9061.

[Steffen Hoffmann]

In 13815:

TagsPlugin: Completing preparation for v0.7 release.

Availability of that code as stable, tagged release
closes #2429, #3359, #3610, #3624, #3677, #3754, #3864, #3947, #3983, #4078, #4277, #4503, #4799, #5523, #7787, #7857, #8638, #9057, #9058, #9059, #9060, #9061, #9062, #9063, #9149, #9210, #9521, #9630, #9636, #10032, #10416, #10636, #11096, #11147, #11152, #11274, #11302, #11658 and #11659.

Additionally there are some issues and enhancement requests showing progress,
but known to require more work to resolve them satisfactorily, specifically
refs #2804, #4200, #8747 and #9064.

Thanks to all contributors and followers, that enabled and encouraged a good
portion of this development work.