Last week we reviewed the operational inputs of the software testing process. This week we should complete the picture by reviewing its outputs.
As part of our analysis we identified six channels that provide input to the testing process (Requirements, Scheduling, Designs, Risks, Defects, and Organizational Feedback); output channels are less in number but bigger in importance and complexity than their input cousins.
The main output channels for the testing process are:
– Testing Results
– Reported Defects
– Formal and Informal Feedback
1. Testing results are the first and most direct output channel.
When you report the scenarios that were tested and the results of these tests you are providing the equivalent of “military intelligence” to the development teams and project managers of your organization, allowing them to make timely adjustments and modifications to their tactical decisions and strategic plans.
The way you write the report is critical:
– Use an intelligent structure; start by giving summarized information with your analysis of the situation and not only the raw data.
– Stay away from endless tables with numbers and test statuses that don’t mean anything to anyone. Include summaries, percentages and convergence vectors with your numerical results.
– Use graphs, trends and extrapolations to help you give information to your stakeholders.
– When creating your reports, make sure each individual gains access to the information he requires: A project manager will want to see application coverage and risk analysis reports, but will not gain value from the status of specific tests. The opposite will be true for the developer in charge of the functionality covered by your test cases.
– Since report generation can take a lot of time, try automating as much of it as possible. Data gathering and graph generation can be quickly done using simple SQLs and tools like Excel Macros or report templates.
I will to leave the subject of test reports in order to move forward with the other outputs, but since this is one of the most important aspects of testing I will review it further on a future post.
2. Reported Defects are also an important output channel, and in some organizations the most widely used.
Don’t underestimate the importance of a good defect report.
– Select a good and comfortable defect management platform, one that will be useful to all stakeholders in your organization and will help you manage all the defects and data.
– Spend the time to create the appropriate template for your defects, one that will help your Organization in 3 major tasks:
(a) Defect categorization and prioritization – help the product team to decide if and when to fix each bug.
(b) Defect fixing – help the developer locate the error and correct it.
(c) Defect statistical analysis – help the QA and Project managers to understand if there are specific process or project issues that need to be corrected.
– An important thing that QA teams forget to include in their bug reports is an indication of the severity of the issue in the eyes of the end user. This information is not always trivial to the stakeholders who are not in direct contact with the application, running test scenarios in the way customers do.
3. Formal and Informal Feedback is the last of the common output channel.
These are all the comments, observations and other information aimed at correcting and/or improving the product or the process.
Examples of such feedback are:
– Usability reports providing user-perspective information about a specific product or feature to the product management or development departments.
– Defect summaries with an aggregated quality overview of a specific area in the application.
– Post-mortem process analysis reviews, enumerating the positive and negative aspects of the process in order to reaffirm or correct them respectively.
– Informal comments to developers and managers providing feedback on a feature or issues that may require further work.
– Cross group informal communication, mutually updating about the latest topics and issues affecting the different teams.
Having seen many testing groups over the years and Continents I’ve reached the conclusion that most teams do their testing job more or less correctly, but nearly all of them fail in the job of communicating the information efficiently and effectively.
At the end of the day, doing the testing and failing to communicate the results is like not performing the tests at all, since the information will not reach the people who need it in order to make the correct decisions at the right time.
Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.
Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.
With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.