Opened 14 years ago

Closed 13 years ago

Last modified 13 years ago

#7486 closed enhancement (fixed)

Customizable test case outcomes (aka verdicts)

Reported by: Roberto Longobardi Owned by: Roberto Longobardi
Priority: normal Component: TestManagerForTracPlugin
Severity: normal Keywords:
Cc: Ryan J Ollos, manolowolder Trac Release: 0.11


It would be cool, if one could define a set of possible verdicts.

You have "Successful", "Untested", and "Failed",

which is OK for many purposes. But if you do tests according to one of the many standards, you might have different needs. E.g. ISO-9646 has five verdicts:

None = no result yet, untested Pass = good, successful Inconc = inconclusive, unclear Fail = bad, failed Error = there was an error in performing the test

If you test according to POSIX 1003.3 you have the verdicts:

PASS = good, successful FAIL = bad, failed UNRESOLVED = inconclusive, unclear UNTESTED = no result yet, untested

Some testing frameworks add XFAIL (= expected fail), UPASS (= unexpected pass) and UNSUPPORTED (= the implementation under test doesn't support a feature) to the POSIX verdicts.

Attachments (0)

Change History (5)

comment:1 Changed 14 years ago by Ryan J Ollos

Cc: Ryan J Ollos added; anonymous removed

comment:2 Changed 13 years ago by manolowolder

Cc: manolowolder added

comment:3 Changed 13 years ago by anonymous

Resolution: fixed
Status: newclosed

Implemented in 1.4.1.

The default outcomes will now be configured in trac.ini at plugin installation (or upgrade) time as follows:

green.successful = Successful
red.failed = Failed
yellow.to_be_tested = Untested
default = TO_BE_TESTED

You can customize the outcomes in any moment by adding new outcomes, or modify the descriptions of the current ones. Do not delete previous outcomes if you have already assigned them to any of your test cases.

For example, to add a new outcome named "Big Mess!!!" to the failures, add a line like the following:

red.bigmess = Big Mess!!!

comment:4 Changed 13 years ago by manolowolder

I have tested the new version, the solution is pretty much what I was looking for.

Status = Successful


comment:5 Changed 13 years ago by Roberto Longobardi

Developer status = happy :-)


Modify Ticket

Change Properties
Set your email in Preferences
as closed The owner will remain Roberto Longobardi.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment

E-mail address and name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.