Testers!! know your Business (and don’t do that of others!)

I want to start this post by thanking the two people who “drove me” to write it.  First of all, Jerry Weinberg, who wrote a comment to an old blog post of mine that helped me realize something I wrote back then was not correct and needed amending.  And also Lalit Kumar, and old friend and colleague, who’s also the editor of Tea Time with Testers, where the blog I mentioned above was published and were Jerry caught sight of the error and wrote his comment.

Life taught me not to keep quiet and always say what I think is right out loud.  Life also taught me to surround myself with intelligent people who will not always agree with me, and who’s (constructive) criticism and (positive) arguments will challenge me to keep growing.  Most importantly life taught me that it’s OK to make mistakes, and it is even better to acknowledge and correct them as soon as I realize about them.

I believe it was Roosevelt who said: “The only man who makes no mistakes is the man who never does anything.”

Jerry & Lalit, many thanks !!

———————–

Some time ago I wrote a blog post called “Principles of good bug reporting“.  It talked about the basic things that make a good bug report and helps testers not to make the typical mistakes that will be criticized by other members in their team.   One of the points I mentioned is to always make sure you are giving bugs the correct Severity & Priority grades, since it is typical of inexperienced testers to give their bugs higher severity with the (sometimes unconscious) aim of having them corrected by the development team.

Jerry Weinberg commented about this blog saying among other things  “I don’t think testers should be prioritizing bugs“; and when I look back at what I wrote I agree with him, I was only half right on asking tester to set their severity and priority grades correctly.

A tester should know his business

The half that I was right was that a tester should be able to set the Severity of his bug correctly.

Severity should be (as much as possible) an objective measure of the damage caused by the specific bug to the Application Under Test (AUT).  And it is something a tester should be able to determine correctly.

The best way for a testers to set the severity of a bug correctly is by understanding enough of his product.
– What does the product do (and not only what is written in our site or in the marketing brochures)?
– Who uses the product and under what circumstances?
– What are the different flows a user can work with the product?
– How knowledgeable are the users in order to find workarounds to specific bugs?
And all the rest of the non-trivial information about the product and its users that will allow him to determine correctly how severe is a specific bug in the system.

I expect every tester, within a logical amount of time and with the correct guidance and information from his team, to reach this level of knowledge.  And the reason she needs to know the product this way is not only in order to set the severity right, this is the only way by which this person will be able to define, design and execute his tests correctly.

A tester should also know NOT to do the work of others in his team

On the other hand, and here is where the criticism from Jerry came, we should not require of a tester to determine the priority of a bug, remembering that priority is related to the time when a bug should be fixed, and it is only partially determined by its severity.

Let’s understand first of all that priority unlike severity is a subjective measure, subjective because it will be determined based on many personal (or team-related) factors and so it cannot be giving outside the context of the whole team.

What factors may affect the priority of a bug:

– Obviously one of the most important factors is the severity.  We will want to fix critical bugs before we solve the less critical bugs.

But there are many other factors such as:

Complexity of the fix, we will want to solve complex fixes before we solve the trivial ones.  We do this in order to assure that a complex fix will be tested more and more thoroughly.  On the other hand sometimes we will choose not to solve a bug because there is not enough time to test it enough and so we prefer to release the bug as is than to solve it and introduce more severe bugs or delay the release of the version.

Available resources.  Sometimes we choose to solve bugs based on the developers who are available, for example solve a simple GUI bug now and leave the complex DB bug for later due to the fact that the GUI developer is leaving on vacation next week, or because we need to send the screens now to the customer for validation.

Bug fix clustering.  It is logical that if we are going to be working on a specific area of the code we try and fix as many bugs in it as possible, and so sometimes we will give bugs priority based on where we are already fixing other bugs.

Earlier promises.  This is one of the most problematic, but also very easy to understand…  We will want to fix first the bugs we already promised to customers, since they are already aware of them and they are expecting them to be fixed.  So if we have 2 bugs with the same severity or even a bug that has a lower severity but was already promised to be fixed to a paying customer, we will most probably be taking care of it before we handle other bugs.

As you see, there are many reasons (all of them good and sensible) why we cannot set the priority of a bug, and why we should leave this to either the person who has all the information or at least make sure the decision is taken by the whole team in order to reflect as many of these points (and also the ones I left out) into consideration.

, ,

13 Responses to Testers!! know your Business (and don’t do that of others!)

  1. Mira Razman May 23, 2011 at 8:33 am #

    I like your fix.  Cheers, Mira

  2. joelmonte May 23, 2011 at 8:48 am #

    Thanks!
    -joel

  3. Lalitkumar Bhamare May 23, 2011 at 9:09 am #

    Dear Joel, 

    Thanks for the mention. A proud moment for me to see myself mentioned on the very 1st QA blog that I have been referring from time when I was  a Kid in S/W Testing. 

    Your humbleness and attitude is one more thing that am learning from this blog now. 🙂

    Thanks and Regards,
    Lalitkumar Bhamare
    http://www.teatimewithtesters.com

  4. Sunita May 23, 2011 at 10:24 am #

    Liked it .

  5. Rikard Edgren May 23, 2011 at 10:28 am #

    I agree that testers shouldn't decide Priority (or severity), but I don't understand what the problem is with testers putting a Priority, that is a recommendation, that can help the ones that decide Priority.

    Many testers know a lot about customers, and a recommended Priority might give a few appreciated fixes that otherwise would have been neglected.

    Testers might make mistakes, but “The only man who makes no mistakes is the man who never does anything.”

  6. JHedges May 23, 2011 at 1:05 pm #

    Joel,
    Thanks for the reminder that its okay for me to say “Sorry, that's not my job.” As a tester, I can tell the developer “this is a big nasty bug”, but its up to the product (or project/development) manager to tell the developer “you should fix this today”. Too often, I get caught in the rut that every bug I find should be fixed as soon as I find it.
    I'm also an advocate of never promising a fix date to a customer. When one of the support technicians comes to me with a bug, they usually press me for a specific fix date. My usual response is that the developer will fix it as soon as feasible, but it may not be released for several weeks. I expect that when I get the fix, it won't be quite right, and that I will have to go back and forth with the developer a couple of times to get things just right.

  7. joelmonte May 24, 2011 at 2:24 pm #

    I think that as you describe in many companies we in the QA get caught in the middle between support (and sometimes sales) and the developers, needing to explain why we have bug X or Y, and sometimes even needing to provide information on WHEN will something be fixed if at all.

    This is not only a hard place to be, but it is also not our responsibility.  I like your approach, it can help other people in that situation.

  8. joelmonte May 24, 2011 at 2:33 pm #

    I don't think there is a problem with suggesting something as long as (1) it is taken as a suggestion, and (b) you have enough grounds to base this suggestion.

    The problem is when either or both of the above pre-requesities are not there and then the tester looses credibility on the rest of the fields (e.g. severity) that should be rock solid.

    BTW, I've worked with some very talented and technical testers that used to go as far as to suggest the coding corrections for the bug, but again this was mostly based on their personal skills and the whole team recognized this fact.

    So in short, there is no “prohibition” but yes a warning.  Don't think you should fill out information that is not up to you to fill and that will harm the rest of your bug report.

    “It only takes a small drop of oil to ruin a brand-new white shirt”

    -joel

  9. joelmonte May 24, 2011 at 2:37 pm #

    Thank you Lalit.

    I don't really see it as humbleness.

    I think that as you become more experienced in any field you can either become afraid other and sometimes younger people notice you are not perfect, and then become an “arrogant expert”.  Or understand you never stop learning and become a “never-ending student”.

    I simply choose the later of the two.

    Cheers.

    -joel

  10. Gerald M Weinberg May 27, 2011 at 6:00 am #

    Well done, Joel!

    And, hey, if you like SciFi, try some of mine. I try to teach testing lessons in my fiction. Makes them easier to swallow for some.

    But I think your articles are even more effective, for the serious tester.

  11. joelmonte May 27, 2011 at 7:01 am #

    Hi Jerry,

    Thanks for the feedback, and I'll be sure to get some of your books and check for the testing lessons in them.

    -joel

  12. Sebastian Giercke June 6, 2011 at 4:14 pm #

    You are perfectly right! 
    Our (the tester's) point is already made by determining the severity of a bug.  The question to be asked is, what is the added information of providing a priority.  When we enter a priority, we only repeat the statement we made by giving the severity, because that is what we can judge on.  Therefore:  We should avoid unnecessary discussions with the programmers or project managers by only providing information within our scope of work.  Maybe later, they might ask for our opinion.

Trackbacks/Pingbacks

  1. Readings « Readings - May 29, 2011

    […] Joel Montvelisky Posted: Testers!! know your Business (& don’t do that of others!)http://bit.ly/kyNBdn based on feedback by @jerryweinberg to an old […]

Leave a Reply

Shares