PractiTest QA Blog

I read a blog by James Whittaker analyzing how for him Software Testing is still stuck in the 90s while the rest of the (technological) world has evolved greatly.  Today I will disagree with Mr. Whittaker since I think we as testers have not thrown all we’ve done away and we have seen real advancement in many areas of our work field.

As he writes in his post, back in 1990 I was a 16 year old high-school student, and as he says I didn’t have an email address (I will say that even in Costa Rica I did have a PC computer running Windows).  To tell the truth I only started doing software testing around 1997, but I still have a clear reference point and can definitely see how the Testing Profession has evolved in these last 10-13 years.

Methodological Testing

We can talk about important methodological advances with the implementation and proliferation of testing techniques such as ET and even some more complex techniques that are still been developed and perfected such as Model Based Testing.

As a tester and a test manager I have seen how we evolved from working as an ad-hoc,  unorganized and (at least in my mind) unprofessional bunch of “button-clickers”; and became professionals in the areas of risk analysis, user advocacy and both structured and heuristically testing techniques.

Not to mention what the current Agile (r)evolution is doing not only to the Development world in general, but to the testing profession in particular.

Testing Tools

When we look back at the subject of the tools we use in testing we can see that many of the “magical-record-replay” solutions that tool makers tried to sell to us felt short from the empty promises made on demos & presentations.

But we have also seen how these same functional automation tools evolved, and how the object recognition approach they were based on became very useful technology.   If you follow this area closely you can see how new approaches such as Keyword Driven Testing provide a more robust solution (still far from perfect), and how we have expanded the reach of the automated testing realm to include more and more business experts to collaborate with our day-to-day work.  Additionally we can see how today we have excellent free tools such as Selenium or Watir (to name only 2!) that make this area more economically accessible to all.

Test Management has also evolved from a science based on excel-sheets and emails, to advanced QA Management Platforms that allow us to seamlessly manage teams and processes regardless if they are all located in a single room, or dispersed over the world in 3 continents and separated by 15 time-zones.

And finally we can look at the advances in load testing tools with the low level analysis that can be reached today with some of the most advanced tools out there, allowing us not only to determine bottle-necks but show the specific areas in the system where they occur.  This is far from the load testing results of “500 users it works – 510 it crashes” that we were able to provide 10 years ago.

The Testing Career & Global Community

Lastly, and to cut it short since we can continue providing examples for a long time we can also talk about the career of the software tester.

What used to be a 1-2 year span before you became a real developer, product manager, or support engineer; became a legitimate career for many excellent professionals who see themselves growing and developing as testers.

Just the increasing amount of professional paths you can take within testing, each with its specialization and challenges, make testing more and more attractive for many engineers that either enter testing directly from school or get into testing after spending time doing development and looking for a more interesting and challenging career path.

Add to this the expanding online tester communities (for example the SoftwareTestingClub) that let you connect, share and learn from many great professionals around the globe and you get additional positive momentum that is already taking our profession higher and higher, day after day!

Maybe its time to turn on the lights

I have no doubt that the overall value we testers generate today is a lot more than that we did 10, 15 or 20 years ago.

So, I think that many of the people who still think we are stuck in the 90s and that most of what we’ve done has been spent and thrown away without really advancing our profession or the way we work today, are suffering from some sort of myopia or lack of perspective.  Maybe a good way for them to come out of the dark ages of testing would be simply by turning on the lights and taking a good look around.

There is a Marketing Saying along the lines of:
“If you do something but you don’t advertise it, it’s like you didn’t do it at all”

This saying is also very accurate in the case of testing, where if you do a test that finds critical issues but then you fail to report them it is as if you had not done a thing (or worst!).

So the trivial lesson is: Find a bug, make sure you report it.

But as Test Managers our problem is more complex than this.  At times, even if we work right and report all our findings to the rest of the team, we see them disregarding our warnings and acting as if we had not said anything to them (e.g. choosing not to fix critical issues, or  releasing a product that is not ready for the market).

We can try to blame the rest of the Organization for failing to listen, but this would be “taking the easy way out”.  I think that most of the times when the rest of the team fails to understand what we are trying to say it is because we didn’t communicate correctly with them.  Or as a friend of mine once said, because our message was “clear as mud“.

Thinking about your audience before writing

Providing information is about telling people what they need to know, when they need to know it, and in a way that will be useful to them.  We need to think about our readers, in this case the rest of the Development or Management Team, and understand what information they need and in what format it will be useful to them.

Many times we write reports as if they were intended to be read by the QA team only.  They include information in a format and style that may not be easy to understand by people outside our team, or that can even be alienating for non-technical people.  In many cases we simply write too much information that is not useful to anyone!

Our external documents and reports need to be formatted based on the needs and customs of our readers, and include the important information in a way that will make immediate sense to them, and will allow them to make decisions quickly and easily without the need to decipher, analyze or in some cases even translate the information from our internal jargon to the non-techie language they understand.

Remember that many times, and specially in written reports, Form is as Important as Content.

Deciding what information should not go into the report

As I started writing above, one of the most important things is to decide what information to include and what to leave out of your reports and documents. Your audience (the people you want to read your stuff!) have limited time and they expect you to choose what information is relevant for their decisions.

People expect you to to understand what stuff is important and what is secondary to the decisions being made.  If you are not sure about this you can consult with one or two members of your audience, and try to understand not only what is important in your current report but what are the principles that make something important so that you won’t need to consult with them all the time.

Value your audience’s time and they will value your opinion.

When to speak up and when to keep quiet

A critical aspect when taking part of a meeting or review process is to decide when to speak up and when to keep quiet.  What you are trying to avoid here is to be taken as the guy (or girl) who is always bringing forward the negatives points on each decision, and when there are not negatives points to make them up just for sport.

Even if your task is to bring forward the risks, bugs and drawbacks you need to make sure to point them out only when relevant and not to fall under the definition of “the boy who cried wolf” in your team.

Provide Information and Stop Being the Gate-Keeper

To summarize, we need to keep in mind that our Job as Testers is to provide the information that will allow the Management Team to make the right decisions.

Management are not very interested in our tests, mostly in our information.  These guys want to know whether the product can be released.  If yes with what risks or potential bugs, and if no why and what needs to be achieved (or fixed) before we can do it.

Hola!
I’m currently in the QA&Test ‘09 conference in Bilbao, Spain.

The conference started yesterday with some tutorials in the morning and presentations in the afternoon, where among other things I presented the concept of Testing Intelligence, with good comments and approvals from the session participants.

I was happy to see that concepts and approaches similar to Testing Intelligence are been developed in parallel by multiple QA Specialist who, same as myself, are realizing that Testing is not only about coverage, bugs, and AUT’s; but about information, stakeholders, and working within a team focused on developing a product.
One of these people is Derk-Jan de Grood from the The Netherlands who gave a great tutorial about Results Driven Testing, with some parallel lines of thought to the ones around Testing Intelligence.

I’m looking forward to a couple of more days of great sessions, starting today with a keynote from Mary Poppendieck…  Will post more in the following days, so stay tuned

It’s been a while since my last post, I’ve been a little busy on a couple of projects and some Testing & QA Seminars. But now we’re back, or not…

I’m actually in Costa Rica, as some of you might know I was born & raised here so this is kind of a friends & family trip but not only that.
The last time I was here a friend commented about the growing local software development industry and this got me thinking, so a couple of months ago I started doing a little research and decided to put together a half-day seminar around Testing Intelligence and some additional topics making waves in the world of testing today.

The “event” took place this last Friday and I was really surprised to see the amount of people who showed up.  We got to fill the conference room with close to 50 people (Testers, Managers, Developers, Analysts, etc) from about 20 different companies (Banks, Development Firms, Outsourcing Companies, Consultants, etc) who were all interested in learning more about how to develop their products more effectively by applying their Tests in a more focused and coordinated way.

For me it was a blast!  To be able to talk about Testing Intelligence and other testing topics with people from Costa Rica was really great; specially to understand that here, in a country that up to now I had linked only to Great Coffee and Breath-Taking Nature there is also a growing community of testers who share the same dilemmas, challenges and issues that I get to see all over the world.

I am really hopping to continue contributing and working with this testing community to help it grow and expand.  As I told them during the session, they are keeping awfully quite and they need open-up to online communities and all the additional sources of information around the Internet…

So here goes for Costa Rica!  Here goes for excellent coffee, for green tracks along the volcanoes & beaches, and for a growing community of IT professionals (and specially their testers!).

Lastly I wanted to thank all the people who helped out on the organization of the seminar, and specially to Melissa Castillo who quietly and firmly made everything happen smoothly.  THANKS!!!

I was taking part on a discussion started by Rob Lambert on the SoftwareTestingClub where he talked about “hitting a brick wall”.  By this he meant the all-too-familiar feeling that regardless how much you raise your voice and warn all the project stakeholders about the imminent danger of following a certain path, they decide against your advice and follow it anyway.

Who hasn’t gone through this before?
I’ve certainly been in the same situation enough times myself, and over the years I’ve learned some tricks and developed a couple of techniques that help me avoid such situations, or at least reduce their number significantly.

1.  I make my homework before the meetings
I always try to talk about the important issues with the project stakeholders One-on-One before the actual meeting where the decision will take place.  This way I can spend the necessary time to thoroughly explain the issues and go over the repercussions it may have for each person and team individually; this also gives them time to think about the issue quietly and ask additional questions if necessary.
If possible I also try to understand from the people I meet what’s their position and thus I know what’s my status before going to the meeting itself.

2.  I make my objective to become the Trusted Adviser of my project Heavy-Weights
This one is easier said than done, but it is the most effective method.
In principle it means that you need to create a reputation for yourself that will make people listen whenever you speak your mind.
How to do this is a science by itself, and it’s definitely not something that happens overnight.
Personally I try achieving this using the principles of Testing Intelligence: “By providing correct and timely visibility into the product and process, that helps my Organization’s Stakeholders make the correct tactical and strategic decisions.”
An additional good approach is also to map your project heavy-weights (the stakeholders who usually carry the most weight in the important decisions) and start by becoming their trusted advisers, as soon as the other stakeholders realize these guys are listening to you they will start doing so themselves.

3. I’m open to change my mind when I understand new things about the problem at hand
Don’t be a hard-head and learn to change your mind, this is the reason I make a point of really listening to the other side when I’m arguing.  One of the dumbest mistakes you can make is to continue fighting for an argument even when everyone realizes you are not right.
The most common cases are when someone presents a new argument you were not aware off (e.g. customer pressure, market reality, etc) and you need to understand that the good of the Company and the Product require the team to take the path you did not support initially.
Smart people need to be smart enough to know when to change their minds…

4. I learned to pick my fights
Since I cannot fight over every decision, I make sure to choose what things are worth fighting for and what battles I need to conceive in order to concentrate on the important stuff.

5.  Learn how to communicate
Maybe the most important, and yet one of the hardest to things to achieve.  We all need to learn how to transmit our messages clearly to the other side.
Just to show a good example of how to do this I will point to another article by Rob Lambert that illustrates one of these techniques (and no, I am not getting any royalties from Rob today…).

To summarize, there are many things you can do in order to confront and even succeed on these kind of situations, and all of them start by recognizing you have additional human beings in front of you.
Try to think what you can do in order to help them understand your argument and your point of view, then you will have no choice but to let them make their own decisions.

There have been multiple debates in places such as QAForums about whether Process and Project Metrics can do more harm than good to the Organization.

My view is that Metrics are primarily good; and most problems come from the way we analyze and display the information.
We also tend to forget that metrics are only an Information Channel and not a product by themselves.

Following are some Rules of Thumb I use when creating a metrics system for an Organization:

(1) Remember that metrics will always be a base for comparisons.
If you know that people will compare the metrics you provide, start by providing the base for comparison yourself.
In some companies this means to provide present numbers vs. past numbers and even vs. target numbers; in other companies this also means to provide information for other teams in the Organization.
Just don’t leave this up to the reader himself…

(2) Metrics need to be balanced.
You cannot measure and show only side of process. Make sure that you display information for all or most activities taking place in the team.
For example, you cannot provide only the number of opened bugs, but you need also to show numbers for Run Tests, Written Tests, and any other activities that measure the activities of the Testing Team.

(3) Metrics need to be normalized.
Make sure you are not trying to balance the outputs of a 15-tester team with that of a 3-tester team.

(4) Metrics need to be comparable.
Following normalization, you need to make sure you are not comparing Apples & Oranges.
Don’t try to compare a team working on an established Enterprise Application with that of a Start-Up team working on a Prototype for a Customer Oriented product.

(5) Don’t make Metrics personal.
Don’t make it a witch-hunt for a single person or a single team to take the blame.
NEVER display metrics for a single individual.
You also need to show the information in a way that the Stakeholders see the truth being displayed and not a way of trying to put blame on subjects.

(6) Think of what you want to communicate by publishing your metrics.
Don’t provide dry numbers only, specially not when you are showing possible problems that need to be corrected.
If you know that stakeholders will immediately ask questions such as “how did this happen?” or “what are we doing to correct this?”, make sure such questions are immediately answered in your report with corrective actions from the team.

(7) Metrics evolve.
It is legitimate and even necessary to review what you are measuring once in a while to make sure you are answering the needs of your organization; if the company’s priorities change from time to time so should the parameters being measured.

Most importantly, metrics are only a COMMUNICATION CHANNEL and not an objective by themselves.
They should provide correct and complete information from one group of Stakeholders to the other in order to allow for the correct decisions to be made.

Remember that you need to serve both sides of the communication process.
The best way of doing this is by not taking any sides, and by making sure both sides know this…