Last week a colleague asked me for help creating a better testing report for her current project. The task got me thinking about how QA managers handle the same information differently; and how this makes the difference between being treated as “Testing Guy” or as the “Inside Information Provider” of a software project.
I’ve learned that when working with people outside our testing teams we need to think like them, understand what information they need and what format will help them understand it faster and better.
Here are 2 simplified examples of different reports:
Example 1. Test execution report
Team A’s Report:
Total Tests in Cycle: 376
Passed Tests: 301
Failed Tests: 28
Tests not Run: 57
Team B’s Report:
Tests in Cycle: 376
Execution percentage: 87.5%
Passed percentage: 80%
Both teams are providing the same data, but who is providing better information? When writing your report remember that people don’t like doing algebra equations in their heads. It is important to understand what information they are looking for, in this case they want to understand (a) how advanced in the testing cycle the team is –> execution %; and (b) what is the status of the application at this point –> passed %.
Example 2. Defects’ report
Team A’s Report:
Total Detected Defects: 453
Closed Defects: 321
Open Defects: 76
Postponed Defects: 56
Total Defects for Release: 397
(Postponed Defects – 56)
Team B’s Report:
Percentage closed: 80.8%
Percentage open: 19.2%
Defect detection rate (2w): 3.2 bugs/day
Defect closure rate (2w): 5.2 bugs/day
Again here, both teams provide the same data but team B also provides more valuable information. Not only did they present the numbers as percentages, but they also point at the convergence status of their project by informing that during the last 2 weeks the fixing rate has surpassed the detection rate.
There are many more examples, but the points I try to remember when defining reports are:
- The less people need to think, the more intelligent my report will appear to stakeholders; as much as possible I provide my data as percentages or rates.
- Anticipate the questions and provide the information up-front. If during the status meetings people always ask me for a specific datum, I start including it in my report.
- Don’t overfill the reports with useless data; too much information will drown the important stuff.
- Using graphs instead of numbers usually provide 3 to 5 times more information.
Finally the most important piece of advice is that a report should talk for itself; the less explanations are needed, means the better you are be doing your job.
