What to pack when you go bug hunting?

Bug Hunting

Bug Hunting is common practice for many testing organizations worldwide, yet some test managers wrongly feel they go Hunting when their testers informally play with the application in order to find “border-case defects”.

What are bug hunts? and what are they not…

bug huntingBug Hunts are Informal Testing exercises; this should not be mistaken as playing with the system without a purpose or objective, Nor should it be confused with ‘Exploratory Testing‘ for the following reasons:

  • Bug Hunts need to be conducted as group activities, while Exploratory Testing can be done by a single tester
  • Bug Hunts are there in order to involve non-testers and find less-conventional bugs, Exploratory Testing is meant to find “regular bugs” and usually involves only testers
  • Exploratory Testing can be used in all stages of the testing process, Bug Hunts require the system to be stable to be useful.

In order to achieve something (and not waste your time!!) during these Bug Hunts, you need to follow a specific methodology, perform planning and preparation actions, and monitor and control the process throughout its execution.

Even if there is no written book (at least that I am aware) on Bug Hunts, I follow a process that I caught some years ago during one of the STAR conferences.

*Update* – I will be presenting and attending the upcoming UKStar event this March (12-13, 2018) if you care to meet and chat “shop” write me.

How to plan a bug hunt

I plan Hunts based on Pair-Testing and Soap Opera Scenarios, combined as a tournament between teams.

Pair Testing is when you take 2 engineers and they work as a team on 1 computer (or environment). The idea comes from pair programming with all its known advantages such as knowledge sharing, brainstorming, and positive pressure from working with a partner. I’ve had very good results pairing engineers from different teams; especially when taking development engineers and paring them with test engineers, leveraging the advantages and points of view from both sides of the process.

Soap Opera Scenarios are relatively short end-to-end scenarios that take the system from beginning to end of a complex operation on a fast (and sometimes exaggerated) chain of events. Working under extreme conditions we are able to find issues that we missed during our controlled scripts and scenarios.

The Tournament activity is where the human factor comes into play. You want people to be motivated, and what better motivation for an engineer than a price… I keep track of many statistics like the number of original bugs detected per day, the most serious bug detected, the worst system crash, etc; and give a price to the pair who exceed on each category (we usually give away t-shirts and in one extreme case we even gave an iPod!), in any case it is not the price but the spirit that makes the difference.

Rules of thumb I give QA organizations before they start planning their Bug Hunts:

1. Make sure you’ve reached a minimal application maturity level; if it is still too easy to detect critical bugs or the system keeps crashing all the time you may be ahead of your time.

2. Work on testing environments with real-life data. Your tests need to be far from sterile in order to simulate a real working environment.

3. Create a balanced activity schedule. I work based on 2-day bug hunts, where each day starts with 4 hours of Exploratory Testing around a specific application area (each team or two working on a different part of the application); and during the second half of the day we run our Soap Opera scenarios, where each team receives a user profile and a list of high-level tasks.

4. Manage the activity using a tracking system. Any bug tracking system will help you keep track of the issues detected by the teams; if possible use a dashboard to make sure all team members are updated on the position of the rest of their peers.

5. Have a bug referee, she will decide whether the findings are real bugs and also stop duplications from been reported and counted.

6. Create anticipation by having internal teasers and “publicity campaigns” ahead of the activity.

7. During the hunt generate an informal atmosphere by playing music, giving each team bells to signal whenever they find and report a new bug, bringing pizzas and sodas, and using other out-of-the-ordinary things that will create the feeling of a special occasion.

Bug Hunts are as much about the right atmosphere and state-of-mind as they are about the methodology and scenarios you run. Make sure you and your team have fun and it will surely generate the outputs your expect it to produce or more.

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.


5 Responses to What to pack when you go bug hunting?

  1. Gustavo Terrera May 19, 2009 at 9:48 pm #

    Is very important this post for me because I’m working in this area: functional testing and exploratory testing.
    Please, ¿Can you send me more information about it?
    Thanks, Gustavo.

  2. Joel Montvelisky May 21, 2009 at 10:58 am #

    Hi Gustavo,
    Just sent you an email, I’ll be glad to send or point you to additional material.
    Just give some more specifications about what exactly you are looking for.

  3. Lourdes July 3, 2009 at 1:59 am #

    Hi, I’m very interesting about this topic I will try to use with my time. But in the company I’m working don`t care the testing process, is the first time they have this process. For me is very difficult to begin to create the important it.
    You can help me ways for this people see the importance.

  4. Joel Montvelisky July 5, 2009 at 12:18 pm #

    Hi Lourdes,

    The importance of testing in many organizations is related to the value provided. This value can come from bugs found in the system that would have been released to users, it can also come from tests that show developers where do they need to continue improving their code, and in other places it is from the information provided to management in order to make their decisions.
    In any case, you need to understand how can you contribute to your process/product/company and start from there.

    I know this is general, but it is very difficult to go into details understanding more about your company and their process.

    You can write to me directly if you’d like to go into this more in details.



  1. Testing Bits – Feb 18th – Feb 24th, 2018 | Testing Curator Blog - February 25, 2018

    […] What to pack when you go bug hunting? – Joel Montvelisky – http://qablog.practitest.com/what-do-you-pack-when-you-go-for-a-bug-hunts/ […]

Leave a Reply