Consider adding a feedback poll to the NewHack template

[Ryan J Ollos]

I find myself spending a lot of time testing hacks before I can even consider implementing them in our production Trac. Often there are a number of similar hacks to accomplish a particular task, and I end up testing more than one.

The number of open tickets is one way to gauge the quality of a plugin. Other useful ways to do this would be to have a poll in the NewHack template, such as the PerforcePlugin#Feedback has.

If download statistics were available on TH.org, that would be another interesting way to determine the popularity of a hack, which sometimes implies quality.

[Ryan J Ollos]

Also, rather than Are you using this plugin?, a more useful poll would be something like:

Poll(Does this plugin work for you?; Yes, it works great!; Somewhat, but it needs more work.; No, it doesn't work at all.)?

This would work even better if it was possible to simultaneously poll for the Trac version that a user was running. It seems as though the PollMacro would need some work to support this, however (​ticket:1895:comment:5).

Nice thing about this is that, it appears from testing that a user can change their vote, so if someone initially select No and the author later fixes the issue, they can change their vote to Yes.

[Adrian Fritz]

Some thoughts

Vote as a way to gauge plugins

• different expectations will drive users to vote positively/negatively but not necessarily reflecting "quality", "how useful", ...;
• current Poll is not time attached, (a change set on Trac or Plugin or both can correct/corrupt some features) then as time goes we get outdated information;
• some plugins (because many factor) will be more used than others, then more visited and maybe more voted;
• for those and other reasons we obtain a very biased result.

I believe

• (plugin) "qualification" must be conduced to be as objective as possible;
• must be a repeatable "process" (to allow comparison, ...);
• it's doable.

Plugin qualification

Some criteria could be based on:

• Plugin Page:
  • it's complete?
    • checklist here (has complete description, list of features, ..., installation, troubleshooting, ... + delta for exceptions);
  • it's clear / easy to understand?
    • written in plain English, follows a style sheet, ...;
  • helps the user to resolve an issue related to the plugin?
• Plugin Source Code:
  • it's complete?
  • ....
• Plugin QA:
  • Encourage use of Sw templates / styles
  • Has passed some automatic test (let's say
  • use of standard accepted fields (instead of creating a new one when already exist one which was set / agreed as 'the reference')
  • Installation: easy?, instructions work?, ​standard process?, ...
• Metrics:
  • Number of open tickets: a high value does not necessarily reflect bad quality
  • has to take in consideration (not an exhaustive list):
    • life-cycle stage for each release, e.g.:
      • while on "definition" or "analysis" could have a lot of enhancement request / new requirements, not bugs
      • while on "implementation" it's expected to decrease the number but still will have many open tickets if developer/users have used ticket system to request / refine feature definitions;
      • at "deployment" a low count is expected;
      • when in "maintenance" period I've have observed plugin with high count but high effort to solve the issues, and low count (with some important issues to be solved) but low activity to solve them;
      • and finally at EOL (let's say a 0.8 / 0.9 branch): probably will cumulate some but probably will not bother;
    • if a new Trac release affected previous and normal behavior;
    • plugin complexity
    • third party dependences
  • works on which Trac versions
  • ....

Bad things

• every release / change the whole process have to be repeated
• for shure there are more...

Finally

• Quality has to be encouraged (Every one wins)
• I endorse the idea of finding and establishing a better way to easy select plugins

[Adrian Fritz]

More thoughts to qualify a plug-in:

• Brainstorm to find metrics:
  • "successful installation" =~ downloads / bug number
    • higher the number of downloads -> could mean higher the interest on such a plug-in -> approximation to number of number of plug-in installations;
    • bug number (ticket type: defect only) -> reflects plug-in maturity, higher the number worst the quality or lack of support
    • both could be obtained automatically (and can have histograms associated also with releases to understand software life-cycle)

[Ryan J Ollos]

Replying to rjollos:

Also, rather than Are you using this plugin?, a more useful poll would be something like:

Strange that the poll inserted into that comment no longer seems to be "open".

[Ryan J Ollos]

Replying to AdrianFritz:

Some thoughts

Some interesting ideas ... probably needing a TracHacksMetricPlugin and a bit of coding.

Ohloh provides some basic metrics (one example is ​here). Some way to enforce a level of install/use documentation, style, and source code documentation would go a long ways towards improving the existing plugins.

If we had at least some documented guidelines (do they already exist, perhaps for the Trac project?), then I would try to follow those as I work on plugins and macros.

[Adrian Fritz]

Replying to rjollos:

Ohloh provides some basic metrics (one example is ​here).

I haven't tested yet. But OhlohBadgeMacro seems to support this metrics on Trac.

• OhlohBadgeMacro: "A small maco for showing Ohloh (​http://ohloh.net) statistics badges."

Source code references / uses that script

12 	    SCRIPT_LOCATION = 'http://www.ohloh.net/projects/%s;badge_js'

[Ryan J Ollos]

Replying to AdrianFritz:

I haven't tested yet. But OhlohBadgeMacro seems to support this metrics on Trac.

• OhlohBadgeMacro: "A small maco for showing Ohloh (​http://ohloh.net) statistics badges."

I've tested the OhlohBadgeMacro with 0.11-stable and 0.12-dev (see #6523) and it works well. I requested installation of the macro on TH in #6473.

I'd be willing to extend the macro to display the other available ​widgets if this was something that would be installed on TH.

[Ryan J Ollos]

Replying to rjollos:

Replying to rjollos:

Also, rather than Are you using this plugin?, a more useful poll would be something like:

Strange that the poll inserted into that comment no longer seems to be "open".

Actually, from looking at the pole on the DoxygenPlugin page, I'm guessing this is a site-wide problem. Perhaps a ticket should be opened for this issue?

[Ryan J Ollos]

Came across this plugin tonight, which might be relevant to implementing features discussed here: DownloadStatsMacro.

[Adrian Fritz]

Replying to rjollos:

Actually, from looking at the pole on the DoxygenPlugin page, I'm guessing this is a site-wide problem. Perhaps a ticket should be opened for this issue?

I realized poll it isn't open when we are not logged in. Otherwise it works properly.

[Ryan J Ollos]

Replying to AdrianFritz:

I realized poll it isn't open when we are not logged in. Otherwise it works properly.

Thanks. Sadly this is not the first time I have been tricked by that, and then completely forgotten the behavior!

[Adrian Fritz]

Brainstorm for NewHackTemplate "Install and Configure" section at NewHackTemplate/InstallAndConfigureProposal?.

[Ryan J Ollos]

Replying to AdrianFritz:

Brainstorm for NewHackTemplate "Install and Configure" section at NewHackTemplate/InstallAndConfigureProposal?.

There is a ticket, #3725, which would be a good place to discuss this.

[anonymous]

Replying to AdrianFritz:

• Metrics:
  • Number of open tickets: a high value does not necessarily reflect bad quality

There was a link to a ​blog article in the SourceForge newsletter this month about the ​statistics they calculate for each project. It might be a useful reference in considering metrics that could be applied to t-h.o.

[Ryan J Ollos]

Adrian: You may be interested in SiteUpgradeProposal, and feel free to add your suggestions to the Wish list.

[Ryan J Ollos]

Good ideas here, but I don't think PollMacro is the right solution.