Stats - Current test status for All Tests wont fit

[anonymous]

Hi there,

first, nice plugin, works rly fine for me and makes my work a lot easier :)

Now for the Problem: If i choose the Stats/Current test Status, with settings set to All Test Plans, somehow the stats sum up the untested test cases, but not the tested ones For explanation: I have 1 Test Catalog with 2 Test Plans running(both including all TCs from the catalog). The Catalog includes ~200 Test Cases. Now the Statistic for all TPs recognizes both TPs use the same TCs, so it says ~200 untested TCs which is absolutely ok for me. But as the test proceeds, i recognized that failed and succeeded tests aren't sumed up, so after both TPs finished, i have like 350 sucessful tested TCs, 50 failed and -200 untested.

I hope my explanations and my english is good enough to bring out the problem :)

greetz and keep up the good work

[podskalsky]

hello,

I've also found that "problem", but i think this is a feature and not a bug! ;-)

I wil try to explain: How should the "All Test Plans" know what to count - If you have the same TC in two or more Test Plans - in one test plan the TC can be ok, in the next it can be untested and in an other failed ... so what should be summerized for "All Test Plans"? I think at the moment the latest state of that TC will be count. (This was the algorythm in my latest view at the source code.)

Maybe Roberto could add an info about that in the WiKi.

BR Andreas

[anonymous]

Maybe this pic will explain what i'm talking about:

​http://i.imgur.com/v5vrn.png

(didn't know where to upload... shows "Untested -170 < 0,01%")

So atm untested TCs ARE summed up, but tested/failed aren't. This means, you can fail/suceed more than 100% of the untested TCs.

[Roberto Longobardi]

Hi, I see your point...

Andreas, unfortunately "anonymous" (:D) is right. Actually there is a bug in the code with how untested TCs are counted.

From stats.py line 219:

        if (not req_content == None) and (req_content == "piechartdata"):
            num_successful = 0
            for tc_outcome in tc_statuses['green']:
                num_successful += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

            num_failed = 0
            for tc_outcome in tc_statuses['red']:
                num_failed += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

            num_to_be_tested = 0
            if testplan_contains_all:
                num_to_be_tested = self._get_num_testcases(beginning, today, catpath, req) - num_successful - num_failed
            else:
                for tc_outcome in tc_statuses['yellow']:
                    num_to_be_tested += self._get_num_tcs_by_status(beginning, today, tc_outcome, testplan, req)

The number of untested TCs is calculated subtracting from the total number of test cases the number of successful and failed ones. The count is right if we want the stats for only one test plan (selecing from the listbox), or if we ask for "All Test Plans" but there's only one test plan. But in case more test plans that share some TCs, the total number of TCs remains constant, while the number of successful and failed ones increases as the user changes the status of any TC.

This problem is not easy to fix. One may think about counting the number of untested TCs, but this is not easy, because when you create a test plan that contains all the TCs in a catalog, no record is actually created in the database (besides the record for the test plan itself), so there ios nothing to count. This is not true when a test plan has only a selected set of TCs in the catalog. This actually creates one record with status "untested" for each selected TC. In fact I'm sure in this case the count is correct.

Mumble mumble.... Do you have any suggestions?

Ciao, Roberto

[Roberto Longobardi]

Cleaning up old tickets.

In the upcoming 1.7.1 release, I've disabled the "current test status" pie chart for "All test plans".

It will display the following message: "'Current test status' is not available for 'All test plans'. Select a test plan to display this chart".