I want to start this post by thanking the two people who “drove me” to write it. First of all, Jerry Weinberg, who wrote a comment to an old blog post of mine that helped me realize something I wrote back then was not correct and needed amending. And also Lalit Kumar, and old friend and colleague, who’s also the editor of Tea Time with Testers, where the blog I mentioned above was published and were Jerry caught sight of the error and wrote his comment.
Life taught me not to keep quiet and always say what I think is right out loud. Life also taught me to surround myself with intelligent people who will not always agree with me, and who’s (constructive) criticism and (positive) arguments will challenge me to keep growing. Most importantly life taught me that it’s OK to make mistakes, and it is even better to acknowledge and correct them as soon as I realize about them.
I believe it was Roosevelt who said: “The only man who makes no mistakes is the man who never does anything.”
Jerry & Lalit, many thanks !!
Some time ago I wrote a blog post called “Principles of good bug reporting“. It talked about the basic things that make a good bug report and helps testers not to make the typical mistakes that will be criticized by other members in their team. One of the points I mentioned is to always make sure you are giving bugs the correct Severity & Priority grades, since it is typical of inexperienced testers to give their bugs higher severity with the (sometimes unconscious) aim of having them corrected by the development team.
Jerry Weinberg commented about this blog saying among other things “I don’t think testers should be prioritizing bugs“; and when I look back at what I wrote I agree with him, I was only half right on asking tester to set their severity and priority grades correctly.
A tester should know his business
The half that I was right was that a tester should be able to set the Severity of his bug correctly.
Severity should be (as much as possible) an objective measure of the damage caused by the specific bug to the Application Under Test (AUT). And it is something a tester should be able to determine correctly.
The best way for a testers to set the severity of a bug correctly is by understanding enough of his product.
– What does the product do (and not only what is written in our site or in the marketing brochures)?
– Who uses the product and under what circumstances?
– What are the different flows a user can work with the product?
– How knowledgeable are the users in order to find workarounds to specific bugs?
And all the rest of the non-trivial information about the product and its users that will allow him to determine correctly how severe is a specific bug in the system.
I expect every tester, within a logical amount of time and with the correct guidance and information from his team, to reach this level of knowledge. And the reason she needs to know the product this way is not only in order to set the severity right, this is the only way by which this person will be able to define, design and execute his tests correctly.
A tester should also know NOT to do the work of others in his team
On the other hand, and here is where the criticism from Jerry came, we should not require of a tester to determine the priority of a bug, remembering that priority is related to the time when a bug should be fixed, and it is only partially determined by its severity.
Let’s understand first of all that priority unlike severity is a subjective measure, subjective because it will be determined based on many personal (or team-related) factors and so it cannot be giving outside the context of the whole team.
What factors may affect the priority of a bug:
– Obviously one of the most important factors is the severity. We will want to fix critical bugs before we solve the less critical bugs.
But there are many other factors such as:
– Complexity of the fix, we will want to solve complex fixes before we solve the trivial ones. We do this in order to assure that a complex fix will be tested more and more thoroughly. On the other hand sometimes we will choose not to solve a bug because there is not enough time to test it enough and so we prefer to release the bug as is than to solve it and introduce more severe bugs or delay the release of the version.
– Available resources. Sometimes we choose to solve bugs based on the developers who are available, for example solve a simple GUI bug now and leave the complex DB bug for later due to the fact that the GUI developer is leaving on vacation next week, or because we need to send the screens now to the customer for validation.
– Bug fix clustering. It is logical that if we are going to be working on a specific area of the code we try and fix as many bugs in it as possible, and so sometimes we will give bugs priority based on where we are already fixing other bugs.
– Earlier promises. This is one of the most problematic, but also very easy to understand… We will want to fix first the bugs we already promised to customers, since they are already aware of them and they are expecting them to be fixed. So if we have 2 bugs with the same severity or even a bug that has a lower severity but was already promised to be fixed to a paying customer, we will most probably be taking care of it before we handle other bugs.
As you see, there are many reasons (all of them good and sensible) why we cannot set the priority of a bug, and why we should leave this to either the person who has all the information or at least make sure the decision is taken by the whole team in order to reflect as many of these points (and also the ones I left out) into consideration.