Keep your mind open when you are testing!

Knowing a bunch of general things, even trivial things, will help you be a better person, a better parent, a better friend, and even a better tester.

encyclopedia crateThis is one of the reasons I listen to a number of different podcasts during my commute to work every day.  I listen to testing podcasts (you can see a list of them on the side of the QA Blog), but I also listen to other podcasts that are not related to testing.

A study of mentoring in the 1930’s

One of the last podcasts I heard is one from Freakonomics Radio called “When Helping Hurts“.   It was about a study commissioned in the 1930s by Dr. Richard Clarke Cabot to prove that mentoring could help kids of low income to get out of the cycle of delinquency and poverty.

Let me make a short break here to say that in the past, and even in the present, I have always loved being a mentor.  I’ve done only a little mentoring for troubled kids, and most of what I’ve done is professional mentoring in testing.  But even on the limited work I did with kids, it used to seem trivial to me that I was doing a good thing for these kids just by being there, helping them with their school work, and trying to be a positive example they could learn from.

Well, back to the Freakonomics podcast and to the study commissioned by Dr. Clarke Cabot.  After a number of years, the study showed that kids who were being mentored ended up in worse a condition than those in the control group who were not mentored at all.

what? surprised expression

Say WHAT?
Did
 you read this correctly?

YES!  Mentoring and the activities done for these kids as part of the study, actually caused the opposite effect of what was supposed to do.  They had a control group, and this control group ended up grading higher in most of the attributes they were checking!

The study and the conclusions are obviously a lot more intricate than what I just described here, and you should really take the time to listed on the podcast or better yet check out the study itself.

But the point I wanted to bring is simple:

Sometimes, even the things that we believe are trivial,
may surprise us greatly!

 

 

OK, cool, but is it related to testing?

Great question, and actually IT IS!

One of the biggest enemies of experienced testers is their own experience.

Maybe to define this better, it is not the experience of the tester per se, but the overconfidence that many times comes from it.

Let me explain.

Have you ever felt that you know exactly what you need to test, and since you have tested this multiple times in the past you can get into auto-testing-pilot?  I know I have, and this is very problematic!

It is even more problematic that most of the times I get away with testing in auto-pilot I am still able to find most of the important bugs in the AUT.

Does this mean that I am a good tester?  Maybe.

But it also makes me even more overconfident, and it is only a matter of time until my luck runs out and I let slip a very big and trivial bug.

 

A tester should never be overconfident

Overconfidence is the biggest enemy of a tester, even worse than lack of knowledge or lack of tools!

Lack of knowledge or tools are obstacles that can be overcome by investing additional efforts and by thinking outside of the box.

But overconfidence is like the thin fog that creeps into our field of vision while driving in a quiet country road.  It makes us believe all is good and “as expected” until a deer crossing the road or a stranded car on the side appears out of nowhere and causes that disastrous crash to happen…

driving in a snow storm

(Sorry for being melodramatic, but I wanted to drive a point home)

If this is the case, we can implement practices or tools to help fight the effects of overconfidence from clouding our testing vision.

Some practical tips to maintain your Testing Edge, even after the experience creeps in

You can try to tell yourself, “I am going to be a professional and not miss any bug in this test“.

But it can be just as effective as when your small kids on Christmas eve say they will stay awake in order to catch Santa Claus bringing the presents from the chimney…

By 10 PM they are sound asleep, just like after 10 minutes of testing you are back to your testing-auto-pilot, thinking about the ski vacation you are planning or the movie you want to see with your friends tonight.

Still, there are some tips and practices you can do in order to maintain your testing edge while running over what may be seemingly simple testing tasks.  Let me go over a few I use all the time as part of my testing practices.

1. Take 2 minutes to plan every testing session you are going to have

Even if you have done the same test a hundred times already in the past, or if you are planning to test this feature for only 10 minutes, sit and plan your testing session.

Think why you are running it?  What changed, and what bugs could this change bring? Plan the main paths you are going to take,  include some that you run all the time as well as some that you have not run in a while, this will help you to add noise to your testing operations and avoid the pesticide paradox.

It is up to you to choose how you want to approach your testing.  You can create a fully defined scripted tests, or keep it lean by writing down some notes on your notebook.

Even if you are going to be running an Exploratory Testing session, make a high level plan or a set of points you want to evaluate, they will serve as beacons along the way.

The vast majority of the times it will better to have a plan and make changes along the way, than to start with no plan at all.

 

2. Test from your Testing Zone

I wrote about how I like working from my Testing Zone when I test.  It helps me to focus and perform tests better.

If you don’t have a zone (yet 🙂 ) at least look for a place where you can focus on the tests and the application at hand, and where people won’t bother you with tasks in the middle.

For those of you working in open spaces, you can create your “zone” by working with really good (or noise canceling) headphones and finding the type of music that lets you concentrate.

I also recommend printing a font 42 page that says something like:
“I am busy testing, interrupt me only if extremely important!”

 

3. Use Heuristics and Testing Resources

There are many resources with heuristics, cheat-sheets, data-sets, and even ideas on things to test depending on the type of application and of testing task at hand.

These testing tools (they are tools if you are using them to facilitate your testing), will not only save you time and remind you of things that you might have forgotten, but they can also provide you new ideas and ways to challenge the system under test better.

Some of the places where I look for my resources are:

https://github.com/ckenst/testing-guides
and
http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf

 

4. Log your testing operations using ET and Session Based approaches

When you are working with scripted tests, make sure to add notes for the temporary steps you check that are not part of the original script.

If you are working using non-scripted tests it is also very important to log the things you tested in a way that will allow you to remember what was tested, as well as to explain the things you covered and found.

I personally like using the Exploratory Testing approach for annotations and use it both when I work with a test management tool as well as when I am taking notes in my personal notebook.

The importance of writing down notes cannot be exaggerated.  First of all, they will help you to remember what you covered in your tests in the future if someone asks if you check for a specific scenario.  But most importantly for me, they allow me to share the tests I did with my peers (developers, testers, POs, etc) and get feedback on things I might have missed during my testing operations.

 

5. Look for things you were not looking for in the first place

This should be instinctive for every “good” tester, but it is important to mention.

Some of the most valuable bugs I ever found came as a complete surprise to me!  What do I mean by surprise? As simple as it sound, I was not expecting to find them and I did not really plan my tests around them, but nonetheless they were there and I was able to detect these bugs.

If you are a good tester and have been doing this for a while, you already know what I mean.  Even when you are doing a specific operation, your brain is open to capture the things on the periphery that are “out of place”.

It also helps to “actively stray away” from the path you are traveling.  The way I like doing this is by defining the borders of what I am testing, and then look for ways to cross over them.

 

6. Review your tests, and not only your results, with peers

I mentioned this before, but it is very important to review your tests with peers in order to find things that may be interesting to check and for some reason you did not think about it yourself.

Many times we plan our tests so that by the time we are done we are also very close to the deadline for our results. This means that when we get feedback for things we should test again it is already too late.

But if as part of your testing approach, you make sure to get feedback on your testing operations with enough time to use this feedback, then you will be able to expand the coverage and the results of your testing operations.

 

The most important thing is not to assume you know exactly what you are looking for!

  • Challenge your assumptions!
  • Don’t take anything for granted!
  • Look for the unexpected!

, ,

Shares