distributedlife

If you have a usable setup for Service Test within Quality Centre then you are going to lose the ability to provide useful metrics. This is a core benefit from using Quality Centre. I’ve discussed previously the tradeoffs for QC/ST setups and the method I prefer:

• Options for structuring HP Service Tests
• Example of recommended HP Service Test structure in QC

The issue is that each test is an action within Service Test and to get the results you need the service Tests executed from Quality Centre. This will put the results file in a consistent location on the QC Server.

From there you can parse the result file to extract each action (test case) and whether it passed or failed.

If you have a logical structure to your service tests then you can build up an XML document that has each service, the endpoints, the operations and the testing aspects. Within each aspect are all the test cases that have been executed.

We can then get the pass/fail rate for each operation/endpoint/service to report to management and development teams.

Generated Structure

I generate the following structure. Every node can be repeated under its parent node. All the attributes listed are all I use.

<services>
    <service name="My.Service.Name">
        <endpoint name="Endpoint One">
            <operation name="GetAwesomePeople">
                <aspect name="Input Validation" start="22/03/2010 - 2:02:17" finish="22/03/2010 - 2:02:23">
                    <case name="Name not supplied" start="22/03/2010 - 2:02:18" finish="22/03/2010 - 2:02:20" result="failed" />
                    <case name="Name is larger than db value" start="22/03/2010 - 2:02:20" finish="22/03/2010 - 2:02:20" result="passed" />
                    <case name="Name is one char" start="22/03/2010 - 2:02:20" finish="22/03/2010 - 2:02:21" result="passed" />
                    <case name="Awesome person does not exist" start="22/03/2010 - 2:02:21" finish="22/03/2010 - 2:02:22" result="failed" />
                </aspect>
            </operation>
        </endpoint>
    </service>
</services>

Service Test Report Structure

The following structure is parsed from the results.xml file that is generated after running a service test project. It’s the same as the QuickTest Pro format.

• /Report/Doc/DIter/Action – each action is a test case
• /Report/Doc/Summary – the execution and pass/fail for the project

Within each Action node:

• /AName is the node that contains a CDATA section containing the action name or test case
• /Summary contains the results of the test case. Because each step in Service Test is of pass or fail significance I only consider a pass to be an action that has a failed count of zero.

For this job to work the executing process needs read access to the Quality Centre file server. This is where the results documents are stored.

Benefits

If you have a automated cycle changer and automated test execution you can generate a report each morning on the tests. How many have passed and failed and which direction each service is going on a daily basis.

Automated tests are useful but their true usefulness comes out when you have them run against every cycle change in an automated fashion. This frees up your service testers to write new tests and as I spoke about before if you structure your tests well you won’t have much work to do each time a breaking change does occur.

I extended this application even further. I have my service testers annotating tests with their expectations and defects that impact the outcome of a test. This information is collected and stored in the XML document. A web site was built to serve the XML document in various ways depending on who was interested in the information.