Are we there yet? Using Bug Charts to determine the completion of your Development Project

Bug Charts – What are they and why do you need them

Bug ChartsEver gone on a long car trip with kids? After some time they start growing restless and begin asking every 5 minutes (if your lucky) ARE WE THERE YET? ARE WE THERE YET? ARE WE THERE YET?!?!?!

I’ve had similar experiences in Development Projects. Close the release date we’d start getting calls from Product & Project Management asking us whether the product is ready to be released and if we can give them a date (even an approximation) for when will we be ready to go ahead. What usually happened in these situations was that Development proclaimed they were finished with all the features and now the QA was halting the project due to the bugs they were discovering in the product (like we were the ones who got them into the code in the first place…)

In my rookie days I used give the neutral answer that “we will be ready once we find and fix all the critical bugs”. Then after a couple of such projects I was able to find a way to give an answer that provided more information by giving visibility into the product via our test results and the statistics related to our bug detection and resolution convergence curves.

The Bug detection chart

My experience is that bug detection behaves like a trend; if 2 weeks ago you detected 15 critical defects, a week ago 12 critical bugs, and this week 10 critical bugs, most probably you will detect between 6 and 8 critical bugs during next week’s testing. Bug detection trends can be represented by a curve that increases at the beginning of the project as you get more features and more people to start taking part of your testing operations, then it reaches a maximum peak when you get all your features and/or once you reach a critical mass of functionality to test, and then decreases as you start completing your testing scenarios and start running your regression tests.
If on top of this behavior we also graph the fixing rate of the development we can then determine 2 things:
1. The time when development starts fixing more bugs than the testing team is finding per week. Even if this point by itself does not provide much vital information, it gives an indication that the project has reach the first turning point toward convergence.
2. By following this graph over a period of time we will be able to provide an intelligent forecast of the approximate time when we will be ready to release the product.

Even if the information on these graphs is not infallible, and even if we should always be prepared for the bug on the final CD before release, by using this data intelligently we are able to provide valuable information to the Organization regarding the time when the product will be ready to be released.

In conclusion…

The main task of the QA team is not to be the Gate Keeper of the R&D, holding the product until it has been completely rid of bugs. Our Job is to give visibility into the process and product, and to provide information to all stakeholders that allows them to make their decisions concerning the product.

By making data our most valuable resource, we make information our most powerful tool!

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

3 Responses to Are we there yet? Using Bug Charts to determine the completion of your Development Project

  1. Anonymous July 3, 2008 at 1:37 pm #

    one small limitation of this method, is that not all bugs are alike,
    And while peak time normally shows a mixture if Medium & Minor bugs, in my experience the last mile normally shows very little amount of bugs, but most of them are Critical.
    Now fixing each of these items, might delay the release in 1-2 weeks, not to mention the amount of regression required…

    So in parallel, we need to consider how to find the Most Important Bugs First.
    (i.e. – In earlier stages)

    Kobi Halperin

  2. Joel Montvelisky July 6, 2008 at 9:36 am #

    Hi Kobi,

    Your comment is correct and thus at times we may like to filter bugs from a certain severity and up when creating these graphs.

    Still during the initial and medium cycles I would leave it “as is” in order to get a good perspective of the amount of issues discovered, keeping in mind that sometimes applications suffered of death by a thousand paper-cuts instead of by one Big Bug…


  1. Do you have Information or Garbage in your Bug Management System | QA Intelligence - a QABlog - April 11, 2013

    […] of the rates for bug detection and resolution you are able to understand whether your project is on-track or slipping out of control for its expected release […]

Leave a Reply