testplan enhancements

[anonymous]

It would be nice to create a testplan with a individual collection of tests. Sometimes it is not useful to test the whole container. The testplan should be something like a template that i can run multiple times.

[Roberto Longobardi]

Well, the only ways to implement this feature that come into my mind would result quite difficult to use...

The testplan should be something like a template that i can run multiple times.

The test catalog is actually the template, and the test plans are the way for you to run the test cases in the catalog multiple times.

Consider anyway that even now:

• You can create a test plan out of any test catalog or sub-catalog. This means you may organize your test cases in sub catalogs in a way that corresponds to your regression test needs. You may then create specific plans only on those sub catalogs that in any particular time you wich to test.
• You may think that when you create a test plan out of a catalog, the plugin makes a copy of all the test cases in the catalog with a status of "untested". This is not true. For optimization purposes, a test plan starts empty, and it references test cases directly from the associated catalog, based on their "path". Only when you set a status to a test case (i.e. successful or failed) an instance of a test case "in plan" is created in the DB. All this just means that you can safely create "sparse" test plans (i.e. where you only plan to execute a small set of test cases out of), with no performance or space impact on the system.

Please, let me know if this makes more clear how things work.

Ciao, Roberto

[anonymous]

Let me explain my problem a little bit:

I have a platform software for different controllers (X, Y and Z) and 3 testcases A, B and C. The controllers have different features and therefore they need different tests. The software supports compiler switches to compile the software platform for the different controllers.

Now a have the following situation:

Controller X: Testcases A and C are needed
Controller Y: Testcases A and B are needed
Controller Z: Testcases B and C are needed

If i try to organize the testcases in sub-catalogs i need something like this:

catalog 1: A and C
catalog 2: A and B (but i have to copy A from catalog 1 to 2 (exactly the same))
catalog 3: B and C (2 copys of existing testcases)

I do not like the redundancy in my testcases.

[Roberto Longobardi]

I understand, stiull I think the solution may be the following:

catalog 1: A catalog 2: B catalog 3: C

Plan 1 on catalog A: "Controller X - Tests A" Plan 2 on catalog C: "Controller X - Tests C"

Plan 3 on catalog A: "Controller Y - Tests A" Plan 4 on catalog B: "Controller Y - Tests B"

Plan 5 on catalog B: "Controller Z - Tests B" Plan 6 on catalog C: "Controller Z - Tests C"

Does this make sense?

Ciao, Roberto

[Roberto Longobardi]

Damn Trac formatting bugs :-)

I understand, still I think the solution may be the following:
Catalog 1: A
Catalog 2: B
Catalog 3: C

Plan 1 on catalog A: "Controller X - Tests A"
Plan 2 on catalog C: "Controller X - Tests C"

Plan 3 on catalog A: "Controller Y - Tests A"
Plan 4 on catalog B: "Controller Y - Tests B"

Plan 5 on catalog B: "Controller Z - Tests B"
Plan 6 on catalog C: "Controller Z - Tests C"

Does this make sense?

Ciao, Roberto

[anonymous]

This is a lillte bit complicated because the 3 testcases are an example. In the real world we have 1-2 thousand testcases. So i would need 1-2 thousand catalogs and may be 500-1000 testplans for every controller. This makes the complete testcases environment useless in my opinion.

[Roberto Longobardi]

I see your point, in practice there is no rule about which tests are needed by which controller, you cannot create catalogs with similar test cases that are either all necessary to a set of controllers, or not.

It seems a bit strange to me, but if this is your situation, then you can either use "sparse" test plans, i.e. plans in which you only run a few test cases (and as I said this does not pollute the database with thousands of test cases "untested", but only with the test cases ou actually run), or you will need this new feature.

I'll try to think about a solution which is usable. It would help me to know if you plan to use a programmatic (i.e. Web RESTful) interface, or a manual way of creating such plans with a subset of the test cases in a catalog. Would it be enough to provide a programmatic interface only, or you will need also a human interface?

Ciao, Roberto

[Roberto Longobardi]

Implemented in 1.4.8.