All of these are important questions, and even if there is no text-book answer for them there are some guidelines you can consider when wondering about this subject.

My thoughts on test sizes were prompted by 2 separate events. This week we are starting a long-range project to re-factor PractiTest’s test automation framework (the one we use to test ourselves!). This time we are making a conscious effort to keep our tests as short as possible and to re-use them as part of more complex test-sets.

I am also guiding one of our PractiTest users on his custom deployment, and we are trying to define how deep and extensive should the tests be in order to answer his needs and constraints. With this customer we are reaching the conclusion that most of the tests should be kept small and high level; but still a limited number of regression scenarios will be written very detailed and covering the system from end to end.

When considering the right size and depth of your test the spectrum is extremely wide.  On the one hand you can write a word-by-word scripted test, allowing you to take ‘any person from the street’ and use him to test an application he hasn’t ever seen.  On the other hand you can give an application to a tester and ask for her “professional opinion” without giving her any indication to what the AUT should do or what you want to check, then come back 3 days later to see what she has to say.

I personally think that both of these extremes are wrong! But there is a large space in the middle to choose from, and many times the right choice will not be easy to find or trivially clear.

I personally prefer my tests to be small and as high-level as possible. I believe that if I take good testers and we give them an introduction on what the AUT should do and who it should serve, then I don’t need to provide them with step by steps cases in order for them to do the job right.

Instead of creating extensive test cases I prefer to user generic check-lists and heuristics to help me go over my application. These tools allow me and my testers to test thoroughly and to find the important stuff; but on the other hand it doesn’t drive us mad by writing kilometric tests with low-level steps that no one really needs and just frustrate testers simply by needing to read them.

There is also an added point to having high-level tests, they force us to think when we test!  I’ve seen that if you give even an experienced tester a very detailed test she will turn into a kind of live-test-automation-tool, performing each and every step but being so focused on it that she doesn’t pay attention to the “interesting scenarios” lying right next to her predefined test.

When your tests are high-level steps or even charters or threats (like a couple of interesting articles I read this week about Exploratory Thread-based testing by the Bach Brothers – here are both by James & Jon) you actually motivate your testers to THINK BIG and take more responsibility for the test execution, overall achieving better results and finding more interesting and severe bugs.

On the other hand, having no tests at all and trusting on the intuition of your testers is a sure recipe for failure and important escaping defects; specially when not all your testers are experts in all the system, and even worst if they need to work under pressure when needing to provide results quickly.

So I guess that at least for me size does matter in my tests! I try to keep them small and smart as possible.

What do you think?

Related posts:

1. Rules of thumb vs. Silver bullets
2. Walking a mile on a Developer’s Shoes – why we should not assume they can test like testers

This break from writing was not intentional and it was not really a break since I was really busy with tons of PractiTest work and some additional heavy-duty chores related to caring for 2 small children during their summer vacations.   But there was also a positive side to this pause since it gave me a chance to reflect on what I’ve written on the blog so far, the subjects I’ve covered and those I haven’t gotten to yet.

I did something interesting, using “wordle” (a very cool app!) I generated a word map of the blog’s content:

Word Cloud for the QABlog

It was no surprise that the word TESTING came so strong and central, but there were also some other words that took an important place and I want to take this chance to review them.


SHARE – Sharing is (almost) always positive, but in Software Development and specially in Agile Teams it becomes one of the biggest success or failure factors.
I am currently reading “Agile Testing” by Lisa Crispin and Janet Gregory and it covers this point broadly with many reasons why sharing and collaboration are two of the most important factors for succeeding in agile processes.  By the way, the book is a great read and I recommend it even to testers who are not part of Agile Teams.


TIME – If I had 3 wishes they would be happiness for my kids, world peace, and a 32 hour day!  Time is one of our most valuable assets and so we need to learn to manage it correctly.
But managing time is not enough, we also need to learn to respect it, both our time and that of our colleagues.  For me personally the biggest time waster is context switching. When I stop what I’m doing to read mails, answer IM’s or phone calls, or even when someone walks up to me to ask “just one small question” it takes me between 5 to 10 minutes to reach the level of concentration I was before the interrupt.  If you do this 2 or 3 times an hour you end up spending half the time just getting to restart your work.  Can you relate to this?


THINK – this is a big one!
Most of us don’t really think as much as we’d like to accept, we are mostly busy reacting to what is going on around us.
To really think we need to take a time-out, breath deeply for a couple of minutes and clear our heads from all the urgent things in order to focus on the important ones (notice that most times the urgent things are not necessarily the most important ones!).  When was the last time you did this??
Whenever we take the time to THINK we use our resources better by investing them on the tasks that are really needed.  It’s a shame we don’t get to do this more often…


DEVELOPERS – these guys and galls are our biggest allies, and yet they’re also the ones with whom we tend to be most in-conflict during our projects.
Our relationship with developers makes me think about the classic book “Men are from Mars & Women are from Venus” by John Grey.  In fact we need to understand that the issues between developers and testers are mainly linked to the fact that we are 2 different species (or at the least belong to 2 different cultures) and in order to communicate and work together we need to understand and respect the principles of the other side, what’s important for each of us, and how each one approaches problems and challenges in his own way.
The key lies in understanding & communicating with your colleagues- just like in all types of relationships.


PERSPECTIVE – one of my personal favorites and closely related to thinking.
When you are in the middle of the forest it becomes too hard to look at the trees.  Perspective allows us to take a new look at the issues we are working on and check for new and interesting stuff even in the places we’ve already visited multiple times before.
Gaining perspective is relatively easy, you just need to fully focus your attention on another subject for some time and then revisit your previous task.  Just by switching context you will be able to see things under a different light.
I use this all the time: when testing to make sure I really covered all the important scenarios; when trying to solve problems by looking for different approaches that may give a better result; and even when in I find myself arguing with a colleague when I take time-off to cool down and think about the stuff that is really important.  Give it a try, and tell me if it worked for you!


Lastly an interesting pairing of words that came out of the random image:
GOOD SCENARIOS – something I have not talked about much in the blog but a subject I think I will write about in the near future


I am hoping that now that summer vacations are reaching their end, and after having released some pretty amazing stuff in PractiTest that we were working on for a number of weeks, I will have some more time in my hands to continue posting more regularly…

Related posts:

1. Testing the image in the mirror
   - or -
   Why a good tester needs to be Self-Critical?
2. When you think you’ve done enough, go for a walk and then think again

When I first saw it on the wall of a friend’s office I thought to myself “Australia is really big!”, then I looked at Europe, America & Russia and started seeing things I had not noticed before.

What happened to me here as I looked at the map and started seeing things differently is called “a change of perspective”, and it is a common way to look for new and interesting stuff in old places or to try to solve problems by looking at them “in a new light”.

The ability to look at things in a new light is one of the most important skills a tester should develop.  It is extremely useful in order to be able to test your product thoroughly from multiple perspectives, and maybe even more important to be able to effectively test the same product time and time again without missing the new bugs.

Simple tips to gain testing perspective:

I have a number of techniques I use in order to gain a new perspective during my testing tasks:

1.  Move to another task for a couple of hours/days.

This one is the simplest.  When you think “you are done” with testing something put it aside for a couple of hours or a couple of days and go to test something else.  When you return to the original project look for scenarios or areas in your product you didn’t test before.
It is important that the “time in between” is spent in a testing-related task and not on a meeting, having lunch, or going home for the night; it’s the context-switch that gives you the shake-up needed to find the old-new bugs.

2.  Do a walk-through of the product to someone else in your team.

Grab hold of a developer or another tester in your team and take him on a walk-through of the application and the areas you’ve tested up to now, ask him for feedback.  What you will be doing is a peer-review of your testing, very effective in providing valuable and cost-effective feedback.
This simple exercise has NEVER failed to provide me with interesting new areas to test.

3.  See a demo of a competitor

A very easy and fast way to get new ideas to test stuff is by seeing a demo of one of the competitor products for your application.  While looking at the demo or even browsing through their site you will start thinking of scenarios and functionality in your system you haven’t tested.

4.  Talk to a user (or Product Manager) and ask him to explain the product to you in his words

This one is similar to looking at your competitor’s site but more focused on your product.  The idea is to pay attention to the way others describe your product and look for things in their “marketing pitch” that you have not covered in your testing scenarios.
Great candidates for this task are your sales-people, who are constantly selling your product by highlighting specific features and scenarios in your AUT.

From my professional experience, perspective is a MUST for all testers, and proactively modifying the light under which we test our AUTs is something we need to be able to do in order to be Great Professional Testers.

Do you have additional tips or methods for gaining a different perspective while testing your products?  Please share them with the rest of us by posting them as comments.

BTW, you can buy the upside-down map from the HEMA site and hang it in your wall as a reminder

Related posts:

1. Pair-Testing a practical solution for getting out of the mud
2. When you think you’ve done enough, go for a walk and then think again
3. 5 Ideas on how “User Profiles” can Improve Your Testing

I hope the people who attended the webinar enjoyed it. I certainly did, specially after we started the presentation and my level of anxiety went down a little

I won’t go into the details of the presentation since you can see all the slides bellow or you can see a recording of the presentation, co-sponsored and hosted by Sogeti.

I did wanted to go over some of the questions I was asked at the end of the presentation, since I found them interesting and worth reviewing a little more in detail.


Question: Do you test during the iteration on the cloud or do you test on in-house servers?  When do you test on the cloud, throughout the iteration or only during regression testing?
Answer: We believe in testing as soon as possible on servers that closely reflect our production environment, that is why all our tests (sanity, functional, regression, etc) are performed on servers and environments hosted on the cloud and using infrastructure similar (or identical) to the one we use to run our products live.
This is always a good practice, but it is vital for SaaS and cloud-based applications, where even a small issue in the configuration of the servers can make very big differences in the behavior of the system and the user-experience of our customers worldwide.


Question: SaaS applications are typically highly configurable i.e. different clients have different options and pages etc.  How would you approach this in terms of regression testing? Do you pick a sub-set of the most widely used pages, and create a dummy client configuration or what would you suggest as a best approach?
Answer: This is truly one of the challenges of testing a configurable and modular QA Management Solution such as PractiTest where our users not only can choose to use different modules of the system but can also configure it on different ways to match their needs.
I find that the best way to handle this challenge is by working with User Profiles, and having different environments set up ahead of time that reflect real-life configurations and customizations of our users.  This approach allows us to work on testing environments that reflect the whole spectrum of configurations and customizations we may have.
We also use a learning approach where on the occasions a new issue is detected on our Production environments we learn from it and find ways to incorporate it into our testing process, many times this involves adding a customization pattern to one of our existing profiles.


Question: Are there any specific challenges associated with being on the cloud (Amazon in your case) in regards to functional or load & performance testing?
Answer: There were some challenges involved with our initial testing steps in the cloud.  We’ve learned a lot and constantly improve the ways we work as we run into new difficulties and learn from them.  For example, I already said that our testing environments sit on the cloud, at the beginning it was very difficult to update these environments with new builds or even specific patches we wanted to test.  With time we realized we needed “something” that would allow the QA to deploy the product on the testing environment without the constant help of Development or IT; so we worked with them to create a set of scripts that would allows us to deploy to our testing environments automatically.
And this is obviously only an example of the many things we now do differently due to our testing environment constraints.


There were some additional questions that were asked on the webinar, but I will leave them for future posts since I believe they require more detailed attention.

In the meantime I wanted to thanks SoftTest & Anne-Marrie for inviting me, Sogeti for providing the logistical help and the hosting, and obviously all the people who attended for their time and their interesting questions and feedback.

If someone has additional questions from the presentation or the slides you can post them as comments and I will answer them gladly!

Related posts:

1. Leveraging Customers Rejections into Your Testing Environment
2. There really is a difference to CAST
3. How we test our SaaS QA Platform

So the question now is how do you get to know your users better?  And taking that even further, how can you as a Tester become The Expert Customer-Advocate of your Company?

I asked myself this question some years ago while working as a QA Manager at a large Enterprise Software firm where we had hundreds of thousands of end-users and we were not sure we understood them well enough.

My team played with some ideas and we came up with a number of actions that helped us to get to know our users better, and leverage this knowledge in order to improve our tests, our data and our overall approach to testing the product.  This information also ended up improving the product, since we became better (more accurate) representatives of our end-users within the Development process, providing feedback during the times it mattered most.

I wrote about this in the past, but I think this is a good time to bring the subject up again and to expand some more on what we did and how you can do it too.

5 Ways to Become an Expert Tester Customer-Advocate

(1) Join the Product Team when they go out to customers

This is the simplest way.  Most companies have a person or a team in charge of gathering information from their users and passing it along to the rest of the organization.  In some places this team is called Product Management, Product Owners, or even Account Managers.

Whatever their title is, go and ask them to join their visits next time they go out to the field to talk to users.  Explain to them your objective: to go and see how real people work in order to test the product more realistically.
If your Product Team has any sense they will more than gladly take you along and even choose the customers that matter most for this purpose.

(2) Help to Provide Technical Assistance On-Site

Some good opportunities can be found during technical interactions such as presales engagements, professional service, or even on-site support visits.  As a QA Engineer you have good and deep technical knowledge of your product and you can offer this knowledge to help other people in your company that need to work directly with customers.

For example look for the next time a sales person has to do a complex demo at a customer and ask to go and help him out. Sales people are always eager to get help from R&D since it takes the burden and the risk from running the demo, and it also makes them look better in front of the client to bring a technical expert with him .

(3) Provide Technical Support Remotely

Even if you don’t get to meet user face-to-face you can still interact with them remotely.  Join support calls together with your customer care team and listen to what users do and what assistance they need as part of their routine work.

By taking part of these calls you will understand what product-operations are not as trivial as the R&D thinks, what working assumptions are correct or incorrect,  and what problems and limitations are the ones that bother users most.

(4) Interview and Learn from the People who are in Direct Contact with Users

As we progress in the list we get further apart from direct contact, but that doesn’t mean the value of the information decreases in any way.  One of the most effective ways of getting valuable information about your users (their likes, dislikes, issues, and wishes) is by asking the people who are in constant contact with them.

Go and talk to Sales people, Support people, Professional Services people, to everyone who is involved with the users and can tell you about them.  Gather the feedback from them and use it in order to generate your own User Profiles.

(5) Read What Users Say About Your Product and Company On-Line

Last but not least, look for comments from your users on-line.  Most products and companies have user forums, support sites, and even communities; even better if they are independently run since they will be impartial.

In these places you will mostly see the 2 extremes: customers who think your product is “the greatest invention since sliced bread”, and those who think it simply sucks!
Filtering out the emotions (and emoticons) and you will be able to understand what is good and what is bad; and if you pay special attention you will even get to understand a little about the why…

What do you do with all this information?

Once you are able to gather inputs be careful not to run too fast and reach quick conclusions.  Take some time to investigate the reasons and to correlate between the information you gathered, look for the patterns that will help you reach the deep conclusions about your user, product, company, etc

Your objective by gathering this data needs to be clear: you want to have a better understanding of your users and to paint a picture in your mind of how they see your product.  What do they like and why?  And what they think needs to be improved and how?

Related posts:

1. QA as Customer Representatives within the R&D
2. Where does it hurt?
3. Business Process Testing – are you doing it already?

One of the most important tools a tester has in his “virtual toolbox” is the use of User Profiles.

What is a User Profile?

A User Profile is the representation of a generic user of your system.  It does not relate to any specific user but to the way you can describe the average user(/s) of your application.
In practice, most real applications have a number of relevant User Profiles such as regular user, managers, system administrators, external users, etc.; and for each of them you will want to create a separate Profile.

When creating a User Profile you will define all the personal and professional traits that are relevant to the work this person does with your application.  For example:
- Years of experience in the field
- Knowledge of your tool, or similar tools
- The way in which he/she works: on a desk in his office, in the field while walking, on a store behind a counter, does the person access the application from his iPhone or BlackBerry?
- Interactions with other users of your tool or other tools with which they need to communicate and collaborate
- You may even want to define some demographic information if relevant such as age, nationality, language, etc

Some companies (and I really like that approach) go as far as creating big posters with the description of their User Profiles, including their Name and even a Picture.  Even though this may sound unimportant and even childish it will help you to relate to your User Profiles and to remember who is who better and faster.

5 simple and powerful things you can do with User Profiles

(1)  Write Testing Requirements and Documentation with User Profiles

User Profiles as part of your requirement and design documents allow you to create more concrete scenarios of what the application should do based on the needs and characteristics of concrete individuals, this in turn will allow you as a tester to find problems faster during your learning and review process.

By providing feedback and comments based on User Profiles you also help others to understand you better and to agree or disagree with you based not on their personal assumptions but on what was already agreed upon by the whole team.

(2)  Test Scenarios based on User Profiles

One of the most powerful things you can do with User Profiles is to leverage them when creating your testing scenarios.
Profiles will allow you to work always based on realistic scenarios, allowing you to test the things that will surely happen in the outside world once you release your product.

You should use Profiles not only to create your positive scenarios but also to think about the wrong things or negative scenarios that your users run into based on their constraints and the ways they work with your product.

(3) Test Prioritization based on Profiles

Once your tests take into account User Profiles it will be easier for you to prioritize them based on their importance, risk and even on how common the scenario is and how many user will fall into a bug located on this area.
You obviously want to run all our tests, but just in case you start running out of time this is an additional way to know what not to test…

(4) Calculate coverage based on User Profiles

User Profiles are a simple and more effective way to communicate your testing coverage to external stakeholders.
For example when you talk to your Marketing or Sales guys, it will make more sense to them if you talk about what you’ve tested and haven’t tested based on User Profiles; it will allow them to understand the implications and the risks faster and easier.

(5) Work with realistic test data based on your User Profiles

Profiles can be used to generate more realistic data and not to use sentences and phrases such as “this is a test” and “hello world”, and data entries such as “asdf’ or “123456″.
When you work with realistic and varied data you have a better chance of running into important bugs that will happen in the field.

In PractiTest we constantly leverage “User Profiles” as one of our main sources for testing.  They’ve proven to be effective and to provide great value for the limited amount of effort needed to create them.

Try them out and let me know how they worked for you.

Related posts:

1. How to use easy to find bug information to improve the quality of your testing
2. Become an Expert Customer-Advocate in 5 Steps
3. Business Process Testing – are you doing it already?

Look back and think about the last time you tried to improve something in your life; when you decided to quit smoking, or when you said that it was time to start going to the gym, or even when you decided to stop drinking too much coffee during the day in the office.
Realizing that you needed to make the change was easy, but the first challenging test you had was to take the initial step and start the journey towards your personal improvement.

Fighting the power of inertia -
it’s not really your fault since it is always circumstantial… NOT!

When you start a testing improvement project you run into this same kind of situation.

You start by realizing that something you are doing is not right and it requires improvements.  You call a number of your colleagues and have a chat with them to consult what needs to be modified and even pen down a high level improvement plan.

Up to now all good & fine, but up to now you also were only working on the theoretical pane of things.  Now is time to move from theory to practice…

So you come in to the office on Monday morning (all improvement projects start on Mondays) and you sit to work according to your plan …  But by around 10:30 AM one of your development colleagues drops by to inform you there is a regression bug on a production project and a fix is coming your way, a fix that needs to be tested and deployed ASAP!

Tuesday comes and you tell yourself that since yesterday there was an emergency today you will start to work based on your improvement plan.  Then while going through your email you see one from your boss letting you know that Scott in the next department is running late in his testing, and you and Sally are going to help him finish his regression suit until the end-of-day tomorrow…

Going back to your original testing-improvement plan; the idea was to take the first half of this week to improve the way you were running your tests, this based on the fact that on Thursday morning you need to start working on the first testable build of your new product.

So Thursday morning is here and you did not have a chance to even start working on the modifications you had planned so there is no choice but to continue working “the old fashion way” and leave the improvement plan for the next time you have a chance to “make it happen!”.

It’s nobody’s fault, after all you really wanted to make the change but circumstances stood in your way and did not let you make all the things you had planned.

Stop thinking about big changes, make small changes and make them all the time

Rome was not built in one day!  Story says it started with 2 brothers fighting and then it took hundreds of years of gradual building to make it a city and even more to form the Roman Empire.
What does this have to do with Testing Improvements?  EVERYTHING!

If your project is already running, chances are you will not have time to make any ground-shattering change, so instead of thinking and planning based on big changes, break your improvement-plan down into gradual baby-step modifications.

The advantage of small changes over big ones is triple:
1.  Smaller changes take less time and efforts so you can make them practically at any time.
2. Your team will find it easier to “swallow” smaller changes than larger ones.  Remember that humans like habits, and people don’t like it when you suddenly come asking them to change what they are doing and start working differently out of the blue.
3. In case your change is not successful and you decide you want to go back to the way things were done before, it is easier and less “problematic” to roll-it-back than if you were talking about a big and revolutionary change.

Make an effort to re-use and improve, instead of constantly re-inventing the wheel.

This is one of the hardest parts of any improvement process, and it is also linked to human nature.  Many times we feel that what we need is something completely new, to take what we have and throw it out the window with a chance to start from scratch and stop “living” with the errors we committed last time around.

This approach is very poetic but not too realistic when working in on-going projects.  Simply enough you won’t have the time to do it, and no manager will allow you to throw away man-years or even man-months of work simply because you want a “fresh-start”.  So the alternative is to take what you have and look at what to improve and how.

In some occasions you may have a case for starting all over again (new technology or some exceptional cost calculation) but don’t build on it…

Map your changes based on the value you will gain from them

Finally the question is how to plan and schedule your changes and improvements.  Here the criteria is relatively simple, you need to look at the list of things you can do and always start by the one(/s) that will provide the higher value in the shortest time span.

I am not talking about *only* focusing on the quick gains while sacrificing the long range value, but if you need to choose between the 2 projects it is better to concentrate on the one that has a shorter time span and will generate value and credibility in a shorter period of time.

Ideally you should try to work in parallel and search for the improvements that will bring you both short and long range gains, but if you cannot find stuff like this then my choice is to go for the quicker and safer wins, since when you start a long range improvement you never know for sure what will happen in the middle and if you will be able to even finish it at all.

Stop thinking about it and start working on it

We all have a list of things we’d like to change.

The best, and maybe the only, way to make it happen is stop thinking about it and start working on it.  There is no reason why improvements should start on a Monday, so if you have something on your mind you might as well start NOW.

Related posts:

1. On-Going Improvement – running a Marathon one step at a time
2. The Pesticide Paradox: how to keep your tests relevant.
3. Good Project Retrospectives

There’s a weird believe that we Testers need to be pessimistic in order to be good / professional / successful.

In a course I impart (where I am required to use slides made by a colleague) there is even a bullet saying that good testers require “professional pessimism” as one of our main traits.

I apologize to the person who came up with this notion, but what a bunch of crap!
Most of the greatest testers I know are Fearless Optimists, and I think this is actually one of the trait that makes them exceptional testers.

Testers need to be optimistic in order to be able to find a huge showstopper bug and instead of saying “This is it, this App is Busted!” and turn to another task, continue our testing task with even more strength to find additional issues that require fixing and make sure the App is released correctly.

A person needs to be first of all an optimist in order to be a dreamer, and only a dreamer can sit with a design document or a screen sketch and come up with user profiles and scenarios that will allow her to test the AUT realistically.

And maybe most importantly, only an optimist can come to work each day to find bugs and look for flaws in the work of others, and do this convinced that he is doing it for the greater good of his team, his friends, and his company.

So next time you hear someone saying that you need to be a pessimist in order to be a (good) tester do all of us a favor and correct this person, and remember to do it with a smile on your face

No related posts.

I read a comment on a newspaper this weekend saying that “Team X” had won a game due to its defense.  This reminded me of a game I saw last week while I was *running* in the gym where 2 college teams were playing and it was obvious that the team that was winning, and not by a short margin, did not really have a better percentage from the field but they were “kicking-ass” on Defense, specially winning almost every single defensive.

If you ever played basketball you know that you need to have talent to play good offense.  But in order to play defense you don’t need talent, you need physical endurance and most importantly you need DETERMINATION.  I learned this in my high-school basketball team where all my friends had talent and I decided I also wanted to be part of the team…

But what the heck has this to do with Software Testing???  Everything!

It is true that you can have a natural talent to “attract bugs to you” whenever you log into your Application Under Test, this will certainly not harm you as a tester.
But a great tester is the person who is 110% determined to do the best possible testing job all the time; who’ll work with all his heart even when the task may be less “glamorous” than the one being done by the developer sitting next to him; and who’ll play with his “eye on the ball” but with his head focused on the objectives and goals of the whole team all the time and specially during the hard times.

I think that in the same way as basketball games are won with points but lost with a bad defense, and the teams who succeed in the long run are those with a strong and stable defense; our companies and products sell in the short run with fast features and cool functionality, but in the long run the ones that succeed are those with a high quality and stable product.

So just like in basketball, you need to keep your eye on the ball but your head focused on the objective of the team; and…

DEFENSE – DEFENSE – DEFENSE!!!!

Related posts:

1. Agile Testing – the side of the story R&D Managers tend to forget
2. Coffee, Nature & Software Testing…
3. Great testers, not so great CVs…

This saying is also very accurate in the case of testing, where if you do a test that finds critical issues but then you fail to report them it is as if you had not done a thing (or worst!).

So the trivial lesson is: Find a bug, make sure you report it.

But as Test Managers our problem is more complex than this.  At times, even if we work right and report all our findings to the rest of the team, we see them disregarding our warnings and acting as if we had not said anything to them (e.g. choosing not to fix critical issues, or  releasing a product that is not ready for the market).

We can try to blame the rest of the Organization for failing to listen, but this would be “taking the easy way out”.  I think that most of the times when the rest of the team fails to understand what we are trying to say it is because we didn’t communicate correctly with them.  Or as a friend of mine once said, because our message was “clear as mud“.

Thinking about your audience before writing

Providing information is about telling people what they need to know, when they need to know it, and in a way that will be useful to them.  We need to think about our readers, in this case the rest of the Development or Management Team, and understand what information they need and in what format it will be useful to them.

Many times we write reports as if they were intended to be read by the QA team only.  They include information in a format and style that may not be easy to understand by people outside our team, or that can even be alienating for non-technical people.  In many cases we simply write too much information that is not useful to anyone!

Our external documents and reports need to be formatted based on the needs and customs of our readers, and include the important information in a way that will make immediate sense to them, and will allow them to make decisions quickly and easily without the need to decipher, analyze or in some cases even translate the information from our internal jargon to the non-techie language they understand.

Remember that many times, and specially in written reports, Form is as Important as Content.

Deciding what information should not go into the report

As I started writing above, one of the most important things is to decide what information to include and what to leave out of your reports and documents. Your audience (the people you want to read your stuff!) have limited time and they expect you to choose what information is relevant for their decisions.

People expect you to to understand what stuff is important and what is secondary to the decisions being made.  If you are not sure about this you can consult with one or two members of your audience, and try to understand not only what is important in your current report but what are the principles that make something important so that you won’t need to consult with them all the time.

Value your audience’s time and they will value your opinion.

When to speak up and when to keep quiet

A critical aspect when taking part of a meeting or review process is to decide when to speak up and when to keep quiet.  What you are trying to avoid here is to be taken as the guy (or girl) who is always bringing forward the negatives points on each decision, and when there are not negatives points to make them up just for sport.

Even if your task is to bring forward the risks, bugs and drawbacks you need to make sure to point them out only when relevant and not to fall under the definition of “the boy who cried wolf” in your team.

Provide Information and Stop Being the Gate-Keeper

To summarize, we need to keep in mind that our Job as Testers is to provide the information that will allow the Management Team to make the right decisions.

Management are not very interested in our tests, mostly in our information.  These guys want to know whether the product can be released.  If yes with what risks or potential bugs, and if no why and what needs to be achieved (or fixed) before we can do it.

Related posts:

1. The art of transforming testing data into project information
2. Testing Outputs that generate added value to the Organization
3. Pair-Testing a practical solution for getting out of the mud