Do you think software testing is still in the Dark Ages? Turn on the lights!!!


I read a blog by James Whittaker analyzing how for him Software Testing is still stuck in the 90s while the rest of the (technological) world has evolved greatly.  Today I will disagree with Mr. Whittaker since I think we as testers have not thrown all we’ve done away and we have seen real advancement in many areas of our work field.

As he writes in his post, back in 1990 I was a 16 year old high-school student, and as he says I didn’t have an email address (I will say that even in Costa Rica I did have a PC computer running Windows).  To tell the truth I only started doing software testing around 1997, but I still have a clear reference point and can definitely see how the Testing Profession has evolved in these last 10-13 years.

Methodological Testing

We can talk about important methodological advances with the implementation and proliferation of testing techniques such as ET and even some more complex techniques that are still been developed and perfected such as Model Based Testing.

As a tester and a test manager I have seen how we evolved from working as an ad-hoc,  unorganized and (at least in my mind) unprofessional bunch of “button-clickers”; and became professionals in the areas of risk analysis, user advocacy and both structured and heuristically testing techniques.

Not to mention what the current Agile (r)evolution is doing not only to the Development world in general, but to the testing profession in particular.

Testing Tools

When we look back at the subject of the tools we use in testing we can see that many of the “magical-record-replay” solutions that tool makers tried to sell to us felt short from the empty promises made on demos & presentations.

But we have also seen how these same functional automation tools evolved, and how the object recognition approach they were based on became very useful technology.   If you follow this area closely you can see how new approaches such as Keyword Driven Testing provide a more robust solution (still far from perfect), and how we have expanded the reach of the automated testing realm to include more and more business experts to collaborate with our day-to-day work.  Additionally we can see how today we have excellent free tools such as Selenium or Watir (to name only 2!) that make this area more economically accessible to all.

Test Management has also evolved from a science based on excel-sheets and emails, to advanced QA Management Platforms that allow us to seamlessly manage teams and processes regardless if they are all located in a single room, or dispersed over the world in 3 continents and separated by 15 time-zones.

And finally we can look at the advances in load testing tools with the low level analysis that can be reached today with some of the most advanced tools out there, allowing us not only to determine bottle-necks but show the specific areas in the system where they occur.  This is far from the load testing results of “500 users it works – 510 it crashes” that we were able to provide 10 years ago.

The Testing Career & Global Community

Lastly, and to cut it short since we can continue providing examples for a long time we can also talk about the career of the software tester.

What used to be a 1-2 year span before you became a real developer, product manager, or support engineer; became a legitimate career for many excellent professionals who see themselves growing and developing as testers.

Just the increasing amount of professional paths you can take within testing, each with its specialization and challenges, make testing more and more attractive for many engineers that either enter testing directly from school or get into testing after spending time doing development and looking for a more interesting and challenging career path.

Add to this the expanding online tester communities (for example the SoftwareTestingClub) that let you connect, share and learn from many great professionals around the globe and you get additional positive momentum that is already taking our profession higher and higher, day after day!

Maybe its time to turn on the lights

I have no doubt that the overall value we testers generate today is a lot more than that we did 10, 15 or 20 years ago.

So, I think that many of the people who still think we are stuck in the 90s and that most of what we’ve done has been spent and thrown away without really advancing our profession or the way we work today, are suffering from some sort of myopia or lack of perspective.  Maybe a good way for them to come out of the dark ages of testing would be simply by turning on the lights and taking a good look around.

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

5 Responses to Do you think software testing is still in the Dark Ages? Turn on the lights!!!

  1. Jim Hazen March 9, 2010 at 6:10 pm #

    As I am one of those ‘niche’ people JW talks about (I started in testing in 1987 on DOS) and have been doing this as my career for 22+ years I can say I agree with JW to a good degree and also support your statements about some of the recent changes in our line of work.

    The main problem still to this day is a misunderstanding & misconception of what it is we do as part of this work. This occurs by both people within testing and outside of it. There is still a stigma and mystery associated with Testing.

    The problem stems from both an academic and industrial line of thought that is faulty. Meaning tesitng is not very well taught in Colleges, or at all in some instances. Thus you have an ignorance by the new breed. Also, because of that same ignorance a lot of older people in industry do not fully understand the testing function. Unfortunately Software is one industry that hasn’t caught the clue like others have. I know, I’ve seen it many many times. That is why I consider part of my job as being an educator for my line of work.

    Now a lot of the techniques that Agile uses (Unit Testing, ET, collaboration, sprints/iterations, etc.) are really nothing new and were in use to some effect before 2000 (actually some of it dates back to the 50’s/60’s). It was just not practiced often or forgotten (especially by the Cowboy programming style on the PC in the late 80’s), and now we are having to “relearn”. Also, this is a give me, the tooling to support the tasks were not there. So the last 10 years has shown improvements from a technology implementation standpoint.

    But process and methods have remained relatively the same. Keyword Driven Automation (known originally as Action Word based automation) was invented in the early 90’s by Hans Bulwalda. The first object recognition methods were in place in tools in the early 90’s, but the idea of a GUI Map or Object Repository didn’t show up until about 1994 or 1995 with Winrunner. In 1992 I was using an automation tool for OS/2 GUI tesitng that talked to the objects (or controls as we called them then) on the window form via its Control ID or Name. There was still R/P, but for those of us who caught on early we were doing things at the object level and building automation functions/frameworks that drove the application and our testing. There was the problem of the R/P snake oil that a lot of us dealt with, but we learned valuable lessons we still call on today.

    So, have there been advances? Yes, but there is still recurring problems from the last 20+ years that we deal with. We are not out of the woods and on the same playing field as our counterparts in Development. That is what will be interesting to see how it advances in the next 10 years.

  2. Joel Montvelisky March 9, 2010 at 6:25 pm #

    Thanks for the insights Jim.

    I also agree that there is a lack of education and understanding about the testing profession, and I’ve also wondered if the solution will come once we start seeing respectable degrees on software testing on some colleges and universities (somehow my gut feeling is that this is still too far away).

    I also share your conclusion that (1) we are definitely not out of the woods, and (2) we are lagging behind our development counterparts.
    On the other hand, it might be my blind optimism (about which I posted on one of my earlier blogs) but I think the par is closing, not so much because devs are getting stuck but because there is a positive and constructive momentum around testing.

    The problem is that we will need perspective and time to judge, and for now we can only talk about subjective impressions 🙂

  3. mahi March 10, 2010 at 3:30 pm #

    Methods of testing manual and automated testing having been improving. So, is the knowledge of folks are who are in the industry. Tools have been improving, we are using selenium for testing enterprise application – we find it very good, same level of satisfaction compared to comercial tools.

    Challenge mainly in software testing I see is expectations, testing is done to verify good quality of what is being shipped. Now, good quality is subjective and debatable for many. Quality is a word that is used in different context, different measures by people, everyone thinks they know but one’s measure of quality is not equal to other.

    Theory works well, when we have few requirements and few software components in product developed – by saying all use cases pass, integration tested, security tested, performance tested. However all of the above become n X n combinations to test properly, and teams apply judgement and some math models to say arrive at what they think is “right” set that needs to be tested to minimize the risk and maximize coverage with given resources and time. However – defects come from delta of n X n minus selected test set, now all the intellect put in testing is questionable:)

    I think we still are far off from where education on software testing needs to be, methodical approach to solve coverage and paths analysis.

  4. Albert Gareev March 15, 2010 at 7:50 pm #

    >>If you follow this area closely you can see how new approaches such as
    >>Keyword Driven Testing provide a more robust solution (still far from perfect)

    Hi Joel,

    I’d like to add a few comments regarding test automation.

    A classic keyword-driven automation framework is a result of evolution towards compact, fixed size test code, and separation of test design from programming by introducing universal instructions. But those instructions are still not abstract enough, as they (at least, some of them) are application-specific.
    I use application-independent model-based test language in my keyword-driven framework, structured in associative trees, not tables.

    Automation of straight value-to-value comparison (some call it “automated checking”) is also a dead-end.
    Instead, Artificial Intelligence methods should be used: heuristic comparison based on data life cycle, Fuzzy Logic based validation, One-to-Many / Many-to-Many relationship based comparisons. Again, how one can expect automation to be smart if it is not provided with ability to use a knowledge base? We have a well-established programming methods for that, like associative programming and defaulting matrices.

    Also, employing test scripts for extended tasks, like information retrieval and delivery, preprocessing and visualization, greatly expands tester’s capabilities: you can try 1000’s of data combinations instead of dozens (if doing purely manually), on big number of environment configuration types, and access behind-the-cover data. Once “Pathfinder” is back with tons of categorized information, any suspicious spots are explored manually. An important requirement for this type of automated exploratory testing is having an established test harness around the application/infrastructure under test, which could be updated from release to release.

    Thank you,
    Albert Gareev

  5. R.S.Saiyed August 3, 2010 at 4:59 am #

    Still people have not good idea about testing.I am agree that there are lack of knowledge regarding testing.Testing is done to verify good quality soft wares and web sites.

Leave a Reply