Modify

Opened 3 years ago

Closed 13 months ago

#9663 closed defect (fixed)

Stats - Current test status for All Tests wont fit

Reported by: anonymous Owned by: seccanj
Priority: normal Component: TestManagerForTracPlugin
Severity: normal Keywords:
Cc: Trac Release: 0.11

Description

Hi there,

first, nice plugin, works rly fine for me and makes my work a lot easier :)

Now for the Problem: If i choose the Stats/Current test Status, with settings set to All Test Plans, somehow the stats sum up the untested test cases, but not the tested ones
For explanation:
I have 1 Test Catalog with 2 Test Plans running(both including all TCs from the catalog). The Catalog includes ~200 Test Cases.
Now the Statistic for all TPs recognizes both TPs use the same TCs, so it says ~200 untested TCs which is absolutely ok for me. But as the test proceeds, i recognized that failed and succeeded tests aren't sumed up, so after both TPs finished, i have like 350 sucessful tested TCs, 50 failed and -200 untested.

I hope my explanations and my english is good enough to bring out the problem :)

greetz and keep up the good work

Attachments (1)

v5vrn.png (5.7 KB) - added by seccanj 3 years ago.

Download all attachments as: .zip

Change History (5)

comment:1 follow-up: Changed 3 years ago by podskalsky

hello,

I've also found that "problem", but i think this is a feature and not a bug! ;-)

I wil try to explain:
How should the "All Test Plans" know what to count - If you have the same TC in two or more Test Plans - in one test plan the TC can be ok, in the next it can be untested and in an other failed ... so what should be summerized for "All Test Plans"? I think at the moment the latest state of that TC will be count. (This was the algorythm in my latest view at the source code.)

Maybe Roberto could add an info about that in the WiKi.

BR
Andreas

comment:2 in reply to: ↑ 1 Changed 3 years ago by anonymous

Maybe this pic will explain what i'm talking about:

http://i.imgur.com/v5vrn.png

(didn't know where to upload... shows "Untested -170 < 0,01%")

So atm untested TCs ARE summed up, but tested/failed aren't. This means, you can fail/suceed more than 100% of the untested TCs.

Changed 3 years ago by seccanj

comment:3 Changed 3 years ago by seccanj

  • Summary changed from Stats - Current test status für All Tests wont fit to Stats - Current test status for All Tests wont fit

Hi,
I see your point...

Andreas, unfortunately "anonymous" (:D) is right. Actually there is a bug in the code with how untested TCs are counted.

From stats.py line 219:

        if (not req_content == None) and (req_content == "piechartdata"):
            num_successful = 0
            for tc_outcome in tc_statuses['green']:
                num_successful += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

            num_failed = 0
            for tc_outcome in tc_statuses['red']:
                num_failed += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

            num_to_be_tested = 0
            if testplan_contains_all:
                num_to_be_tested = self._get_num_testcases(beginning, today, catpath, req) - num_successful - num_failed
            else:
                for tc_outcome in tc_statuses['yellow']:
                    num_to_be_tested += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

The number of untested TCs is calculated subtracting from the total number of test cases the number of successful and failed ones.
The count is right if we want the stats for only one test plan (selecing from the listbox), or if we ask for "All Test Plans" but there's only one test plan.
But in case more test plans that share some TCs, the total number of TCs remains constant, while the number of successful and failed ones increases as the user changes the status of any TC.

This problem is not easy to fix. One may think about counting the number of untested TCs, but this is not easy, because when you create a test plan that contains all the TCs in a catalog, no record is actually created in the database (besides the record for the test plan itself), so there ios nothing to count.
This is not true when a test plan has only a selected set of TCs in the catalog. This actually creates one record with status "untested" for each selected TC. In fact I'm sure in this case the count is correct.

Mumble mumble.... Do you have any suggestions?

Ciao,
Roberto

comment:4 Changed 13 months ago by seccanj

  • Resolution set to fixed
  • Status changed from new to closed

Cleaning up old tickets.

In the upcoming 1.7.1 release, I've disabled the "current test status" pie chart for "All test plans".

It will display the following message: "'Current test status' is not available for 'All test plans'. Select a test plan to display this chart".

Add Comment

Modify Ticket

Action
as closed .
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from seccanj. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.