PractiTest QA Blog

I read a blog by James Whittaker analyzing how for him Software Testing is still stuck in the 90s while the rest of the (technological) world has evolved greatly.  Today I will disagree with Mr. Whittaker since I think we as testers have not thrown all we’ve done away and we have seen real advancement in many areas of our work field.

As he writes in his post, back in 1990 I was a 16 year old high-school student, and as he says I didn’t have an email address (I will say that even in Costa Rica I did have a PC computer running Windows).  To tell the truth I only started doing software testing around 1997, but I still have a clear reference point and can definitely see how the Testing Profession has evolved in these last 10-13 years.

Methodological Testing

We can talk about important methodological advances with the implementation and proliferation of testing techniques such as ET and even some more complex techniques that are still been developed and perfected such as Model Based Testing.

As a tester and a test manager I have seen how we evolved from working as an ad-hoc,  unorganized and (at least in my mind) unprofessional bunch of “button-clickers”; and became professionals in the areas of risk analysis, user advocacy and both structured and heuristically testing techniques.

Not to mention what the current Agile (r)evolution is doing not only to the Development world in general, but to the testing profession in particular.

Testing Tools

When we look back at the subject of the tools we use in testing we can see that many of the “magical-record-replay” solutions that tool makers tried to sell to us felt short from the empty promises made on demos & presentations.

But we have also seen how these same functional automation tools evolved, and how the object recognition approach they were based on became very useful technology.   If you follow this area closely you can see how new approaches such as Keyword Driven Testing provide a more robust solution (still far from perfect), and how we have expanded the reach of the automated testing realm to include more and more business experts to collaborate with our day-to-day work.  Additionally we can see how today we have excellent free tools such as Selenium or Watir (to name only 2!) that make this area more economically accessible to all.

Test Management has also evolved from a science based on excel-sheets and emails, to advanced QA Management Platforms that allow us to seamlessly manage teams and processes regardless if they are all located in a single room, or dispersed over the world in 3 continents and separated by 15 time-zones.

And finally we can look at the advances in load testing tools with the low level analysis that can be reached today with some of the most advanced tools out there, allowing us not only to determine bottle-necks but show the specific areas in the system where they occur.  This is far from the load testing results of “500 users it works – 510 it crashes” that we were able to provide 10 years ago.

The Testing Career & Global Community

Lastly, and to cut it short since we can continue providing examples for a long time we can also talk about the career of the software tester.

What used to be a 1-2 year span before you became a real developer, product manager, or support engineer; became a legitimate career for many excellent professionals who see themselves growing and developing as testers.

Just the increasing amount of professional paths you can take within testing, each with its specialization and challenges, make testing more and more attractive for many engineers that either enter testing directly from school or get into testing after spending time doing development and looking for a more interesting and challenging career path.

Add to this the expanding online tester communities (for example the SoftwareTestingClub) that let you connect, share and learn from many great professionals around the globe and you get additional positive momentum that is already taking our profession higher and higher, day after day!

Maybe its time to turn on the lights

I have no doubt that the overall value we testers generate today is a lot more than that we did 10, 15 or 20 years ago.

So, I think that many of the people who still think we are stuck in the 90s and that most of what we’ve done has been spent and thrown away without really advancing our profession or the way we work today, are suffering from some sort of myopia or lack of perspective.  Maybe a good way for them to come out of the dark ages of testing would be simply by turning on the lights and taking a good look around.

There’s a weird believe that we Testers need to be pessimistic in order to be good / professional / successful.

In a course I impart (where I am required to use slides made by a colleague) there is even a bullet saying that good testers require “professional pessimism” as one of our main traits.

I apologize to the person who came up with this notion, but what a bunch of crap!
Most of the greatest testers I know are Fearless Optimists, and I think this is actually one of the trait that makes them exceptional testers.

Testers need to be optimistic in order to be able to find a huge showstopper bug and instead of saying “This is it, this App is Busted!” and turn to another task, continue our testing task with even more strength to find additional issues that require fixing and make sure the App is released correctly.

A person needs to be first of all an optimist in order to be a dreamer, and only a dreamer can sit with a design document or a screen sketch and come up with user profiles and scenarios that will allow her to test the AUT realistically.

And maybe most importantly, only an optimist can come to work each day to find bugs and look for flaws in the work of others, and do this convinced that he is doing it for the greater good of his team, his friends, and his company.

So next time you hear someone saying that you need to be a pessimist in order to be a (good) tester do all of us a favor and correct this person, and remember to do it with a smile on your face

I started working on an automation project in a company where they recently adopted a strong and healthy agile development approach.  One of the first things they did was to distribute the once-unified testing team and incorporate the testers as part of the organic development teams; up to now all good and fine.

The VP R&D, who I know from a previous company we both worked together ages ago, brought me in to solve a problem that started when each of the teams began developing their test automation, all in their own separate ways, and each one started running into different issues halting their progress almost simultaneously.

I will be sincere and say that in my own egocentric way I thought the answer would be trivial and based on the models I had used in the past, where you create a central team in charge of automation and they provide services to the rest of the teams in the most professional and effective way.

Sounds logical, right?  At least it did to me.
So here is where I came (to save the day!?!?!).

The first thing I did was to set expectations.
I am only a Testing Guy with some experience, not a Magician, Miracle Worker, or a Messiah that will solve all the issues by imploring the help of any Testing Deity above (or even bellow).  The best I can do is try to use some of my experience and a lot of common sense (both mine and theirs) to find the way to work correctly.  I hope they understood this, if not the road is going to be really bumpy…

The second thing I did was to list the stakeholders within each team and across the company that I should talk to in order to get a good and balanced image of was is going on.

Here is where it started getting interesting, as things usually do when you start going deep into the bits and bites and asking some interesting questions.  BTW, for this I really recommend using an approach based on the 5 Y’s.

I quickly realized 2 main things, a bad one and a very good one, that were happening in the teams.

The bad thing(/s):
Since the automation had suddenly become a tasks of the Development Team Leaders, and since the Company did not have a single person who had ever been part of a Test Automation Process, they were running into all sorts of “rookie automation” issues.

• Wrong test organization
• Teams not sharing or even aware of the tests being developed by the other teams
• Lack of a proper test management or execution platform
• Lack of test development experience
• Tests that were not written in a modular way
• Tests that, when they fail, don’t give enough information about the nature of the error
• etc.

Just think about all the possible “simple” mistakes in the book, and most probably they were there in one team or another.

The (very) positive thing I saw was that the Team Leaders, who are all R&D managers in one way or another, really took ownership of the testing tasks and their automation.  Not only did they take ownership, but gave it a very high priority within the tasks of their teams, so much so that they were even giving automation development tasks to their brightest developers and team members.

Here is where I realized that I needed to let go of the traditional approach I had in mind.

Even though my initial feeling was to go by the traditional way and solve the issue by creating a centralized team to handle the test automation, I decided that this approach would very much eliminate the involvement of the Team Leaders who had really done the mental switch to adopt testing as part of their core responsibilities.

So, this week when I came back to the VP R&D with my analysis and recommendations I actually presented him a proposal that will take me out of my “known & comfort zone” but will allow us to leverage the great agile spirit currently present in the company.  Instead of creating a team that will develop automation, I will start working with each of the teams in order to solve their issues, and we will create an “informal” automation forum that will be in charge of correlating, sharing, and levering the automation process between the teams.

We will start working in the next 2 weeks so wish me luck!
I’ll keep posting updates and interesting stuff as they will surely become available.

I love making weird connections between apparently unrelated stuff.  This morning I was watching a Monty Python short in YouTube and it reminded me of one of the most important lessons in group communications.

So here is the short – http://bit.ly/daIENg

And for those who didn’t make the automatic connection, the important communication lesson is to understand how to pass along information to each of our Stakeholders based on their specific needs and communication channels.

In our case this means to know what information is useful (bugs, tests, coverage, risks, etc?) and in what format is the appropriate (graphs, tables, written reports, dashboards, etc) for each one of our Main Stakeholders.

And remember to always look on the bright side of life…

Let’s put aside the subject of Testing for a second and talk about more serious stuff – Playing Games!

I had not played Sudoku until about a month ago when I was stuck in a boring meeting and the only thing left in my BlackBerry (after being done with mails) was the Games Menu.

By elimination, and since I had never played it before, I decided to give Sudoku a chance. I choose the MEDIUM LEVEL, after all I was only trying to understand how it worked, and bum!
I got hooked.

Putting aside the addictiveness of the game itself, there are some pretty good attributes to Sudoku that are fundamental to every tester out there.
Naming only a few:
- Analytical pattern recognition
- The ability to look at what is there, and concentrate on what is missing
- Focusing at a problem from multiple angles until you find the thing you were looking for
- Even how to handle sisyphic tasks that not always end in success

There are many games that can be used in order to evaluate and even develop your professional abilities as a tester.

For example, “Where’s Waldo?”
I remember playing this game as a kid. It’s all about developing your eye’s capacity to find a needle in a hay stack!

You can even play Texas Hold’em poker on the Internet.
Weighting your hand while you try to figure out your opponents’ chances based on the open cards on the table helps you to develop your analytic capacity to extrapolate (and take risks!!!) under pressure.

There are many more games that can help you and your team do a better job as testers.
Remember that testing doesn’t need to boring or monotonic all the time…

There is a Marketing Saying along the lines of:
“If you do something but you don’t advertise it, it’s like you didn’t do it at all”

This saying is also very accurate in the case of testing, where if you do a test that finds critical issues but then you fail to report them it is as if you had not done a thing (or worst!).

So the trivial lesson is: Find a bug, make sure you report it.

But as Test Managers our problem is more complex than this.  At times, even if we work right and report all our findings to the rest of the team, we see them disregarding our warnings and acting as if we had not said anything to them (e.g. choosing not to fix critical issues, or  releasing a product that is not ready for the market).

We can try to blame the rest of the Organization for failing to listen, but this would be “taking the easy way out”.  I think that most of the times when the rest of the team fails to understand what we are trying to say it is because we didn’t communicate correctly with them.  Or as a friend of mine once said, because our message was “clear as mud“.

Thinking about your audience before writing

Providing information is about telling people what they need to know, when they need to know it, and in a way that will be useful to them.  We need to think about our readers, in this case the rest of the Development or Management Team, and understand what information they need and in what format it will be useful to them.

Many times we write reports as if they were intended to be read by the QA team only.  They include information in a format and style that may not be easy to understand by people outside our team, or that can even be alienating for non-technical people.  In many cases we simply write too much information that is not useful to anyone!

Our external documents and reports need to be formatted based on the needs and customs of our readers, and include the important information in a way that will make immediate sense to them, and will allow them to make decisions quickly and easily without the need to decipher, analyze or in some cases even translate the information from our internal jargon to the non-techie language they understand.

Remember that many times, and specially in written reports, Form is as Important as Content.

Deciding what information should not go into the report

As I started writing above, one of the most important things is to decide what information to include and what to leave out of your reports and documents. Your audience (the people you want to read your stuff!) have limited time and they expect you to choose what information is relevant for their decisions.

People expect you to to understand what stuff is important and what is secondary to the decisions being made.  If you are not sure about this you can consult with one or two members of your audience, and try to understand not only what is important in your current report but what are the principles that make something important so that you won’t need to consult with them all the time.

Value your audience’s time and they will value your opinion.

When to speak up and when to keep quiet

A critical aspect when taking part of a meeting or review process is to decide when to speak up and when to keep quiet.  What you are trying to avoid here is to be taken as the guy (or girl) who is always bringing forward the negatives points on each decision, and when there are not negatives points to make them up just for sport.

Even if your task is to bring forward the risks, bugs and drawbacks you need to make sure to point them out only when relevant and not to fall under the definition of “the boy who cried wolf” in your team.

Provide Information and Stop Being the Gate-Keeper

To summarize, we need to keep in mind that our Job as Testers is to provide the information that will allow the Management Team to make the right decisions.

Management are not very interested in our tests, mostly in our information.  These guys want to know whether the product can be released.  If yes with what risks or potential bugs, and if no why and what needs to be achieved (or fixed) before we can do it.