Testing Certification Training – the Good, the Bad & the Ugly

By on July 8, 2009 in Management, Test Process

A couple of weeks ago I had the chance to teach another Testing Certification Course in Tel Aviv.
The dynamic of this specific group together with the class discussions & exercises helped me to articulate some of the positive and negative aspects of all these certification courses & exams.

Rob Lambert made a nice classification of the testing world based on their stands with regards to certifications. He identified 3 camps:
1. Those who hate certifications.
2. Those who think some certifications are worthwhile (but they may also be dangerous).
3. Those who are all in favor of certifications, thinking they have the power to generate “Testing Superstars” over-night.

So it should be no surprise that I count myself in Camp number 2, thinking many of today’s Testing Certifications provide good value to the Testing Community.  But I also think there are misconceptions and dangers associated with testing certifications that need to be taken into account both by the people taking part of the courses (and the exams) and by the employers looking for certified testers.

The Good

1.  Certification courses can help testers who have not had the chance to formally learn testing in any other framework to go over all the basics and principles.  This is true for many courses and not necessarily for those related to certifications, but certifications provide a good excuse and incentive to take the course in the first place – something that may not take place otherwise.

2. Certification sillabi are a synthesized source of information prepared by knowledgeable professionals, and this can help engineers who want to get a lot of information in one place coming from a trustable source.  Just make sure to double check what certification you are taking and who composed the syllabus, lately there are many groups offering courses and diplomas that are not really what they promise up front.

3. Only once in a while we have the chance to learn theoretical tools that are not 100% linked to our day-to-day work.  This experience may help us in the future or even help us look at the things we do under a different light.  For me there is nothing like an intensive data transfer session to trigger out-of-the-box thinking.

4. Certifications are a great way of unifying terminology.  This may seem trivial or less important, but once you start communicating with people outside your organization terminology starts becoming a problem.  The fact that 2 people may be talking about the same thing under different names is sometimes the source of misunderstandings and even conflicts.

The Bad

1.  Some students think that theoretical knowledge alone will make them testing superstars and disregard the fact that testing is a discipline that takes time to master.  People who think they can learn all there is from a certification or course are like drivers who feel ready to enter the highway after learning to drive from a printed manual.

2. Teachers who don’t master the art of testing can hurt their students.  Certifications are nothing more than a syllabus, an exam and hopefully a diploma you can hang on the wall.  The real value comes from the training and this part can be jeopardized by a teacher who doesn’t really pass along the principles of testing to his students.  This becomes more acute as we see more and more institutes who are aiming at preparing their students to pass the exam and not to learn about testing.

3. We are starting to see employers who make certifications a blind prerequisite for testers to work in their company.  This is wrong since it closes the door on many good testers who either have not had the time or the money to take the certification, or who don’t need it since they already know all the *stuff* without having the diploma.
I want to share with these recruiters and managers something out of the ISTQB syllabus (not textual, but in essence): “Absence of bugs does not demonstrate the software has Good Quality”; under this same principle Absence of a Certification does not indicate the candidate is not qualified to be an excellent tester!!!


The Ugly

This is something that is starting to bother me more and more…  Some people demonize certifications and classify people who support them (like me!) as evil.

As I stated above, certifications are not magic wands and the most important assets of a tester are his skills, experience and approach to testing (things that cannot be thought on a course or measured by an exam!); but this is far from saying that certifications and their preparation courses are wrong or counter productive.

We need to look at certifications and their training courses as tools, and understand that tools can always be used for good and wrong purposes.  At the end of the day we are talking about humans, the same creatures who discovered fire and after creating civilization with it found a way to create bombs to erase it from the planet…

About Joel Montvelisky

View all posts by Joel Montvelisky →
  • http://www.clarotesting.com James Christie

    I pretty much agree with that. I find the tone of the debate between the pro-certification lobby and the anti lobby increasingly tiresome.

    Certification training is just one step, and not an essential one, on the route to becoming an expert. What’s essential is an eagerness to learn and to push back the boundaries.

    I really don’t buy into the idea that certification training narrows the mind. Quite the opposite, provided (and it is a big proviso) that students approach the training with the attitude that they are merely extending their knowledge and are not going to learn everything worth knowing about testing.

    If they do have that attitude then that is dangerous. However, people with such an attitude are likely to have problems, and cause problems, regardless. I am more concerned by people who work for large companies who assume that their employers are “best of class”, “state of the art” or “industry leaders”, that their corporate model is the only credible one and that there isn’t much to learn from outside. I’ve been dismayed at the insularity to be found in big companies.

  • http://thesocialtester.wordpress.com/ Rob Lambert

    Joel,

    Nice blog indeed and always good to see a view point from someone who teaches these certifications as well.

    I like
    ““Absence of bugs does not demonstrate the software has Good Quality”; under this same principle Absence of a Certification does not indicate the candidate is not qualified to be an excellent tester!!!”

    So true.

    I reckon certifications will always be a tricky subject to address particularly as more and more companies are simply using them as a yardstick for measuring competence. It does feel though that although there is an uptake in certifications that the industry as a whole is realising that manual testers add value that is not possible to teach or measure.

    It’s a nice feeling that actually testers are being considered for their attitude, critical thinking, passion and communication skills rather than a certification. On the other hand, those new to the business will no doubt benefit from an introductory course and certification in software testing.

    I’m firmly on the fence.

    P.S – Thanks again for the reference!

    Rob..

  • http://www.practitest.com Joel Montvelisky

    Thanks Rob!

    I also think the Industry is starting to learn how to “eat” certifications and what they really mean to a tester.
    There’s a long way to go, but at least it is encouraging to see the first steps…

    -joel

  • http://www.practitest.com Joel Montvelisky

    Thanks James,

    Totally agree that it is in the hands of the student/tester to take the knowledge to the places that will help him grow and develop.

    Large companies will usually think they are doing things right, and I think that people inside them are the only ones with the power to make the changes. From my experience having worked in one, it takes time and not a small amount of pain but with the right attitude it can be done…

    -joel

  • http://www.clarotesting.com James Christie

    I should really have added that it’s also vital that trainers don’t give the impression that what they are teaching is “everything worth knowing about testing”. That was rather an unspoken assumption. I’ve not seen it, and I doubt if it’s widespread, but I dare say that some training providers do sometimes go too far.

  • http://www.practitest.com Joel Montvelisky

    I agree. As I trainer I make an effort to point out the value of self learning and the need to continue looking for new sources of knowledge and information.

  • http://www.softwaretestingclub.com/profile/AinarsGalvans Ainars Galvans

    Although there are no “standardized contents for education”, it’s not uncommon that ANY University diploma as blind prerequisite to get a job. I don’t think it is so bad. Why?
    Teachers in my university used to say that knowledge is not what we get from university. We learn how to acquire knowledge.

    I think the only bad thing is people who don’t position education as something that you will benefit from through all your life, but rather shows short-term benefits such as get a job or hire people with the same background…

  • http://www.practitest.com Joel Montvelisky

    Ainars,

    I don’t think standardized education is bad, the best proof of this is the fact that I impart such courses.

    What I think is bad is to have any sort of blind prerequisites to get such things as a Job. I know a number of great testers who did not get a University degree and the fact that they became testers in the first place is because someone gave them a chance.

    I agree with what your teachers said, it is not scalable to think that only through frontal training of any sort you can gain knowledge.

  • http://josephours.blogspot.com Joseph Ours

    When it comes to testing certifications I am in Camp#1. The ones that are currently there are loathsome to say the least. There is a lot to say on this topic, but I would like to address your “Good” points outlined above. I’ll limit this discussion to the ISTQB certifications since that is the parent organization of the ITCB to which Joel belongs.
    1. Certification courses can help testers who have not had the chance to formally learn testing in any other framework to go over all the basics and principles.
    First, as you point out, certification is not required for people to learn. In fact, learning can never be achieved via certification. Even the ISTQB certification is a 40 question multiple choice test. It doesn’t teach. If testers need a good excuse to learn, then they aren’t very good testers any way.
    2. Certification sillabi are a synthesized source of information prepared by knowledgeable professionals, and this can help engineers who want to get a lot of information in one place coming from a trustable source
    The ISTQB is anything but a trustable source. In fact the ISTQB’s mission statement says It is the ISTQB’s role to support a single, universally accepted, international qualification scheme, aimed at software and system testing professionals, by providing the core syllabi and by setting guidelines for accreditation and examination for national boards
    So, their mission is to create a certification scheme (their word, not mine) and provide the materials and exams for that certification. It is nothing less than a money making scheme, and it is right there in their mission statement. They do not care about quality or testing. I’ve written more about this here
    Only once in a while we have the chance to learn theoretical tools that are not 100% linked to our day-to-day work.
    Can’t this occur without a taking a certification test? Again, certifications do not teach, you can’t learn from them.
    4. Certifications are a great way of unifying terminology. I’ve heard this argument many times before. So is a dictionary and you don’t have to take a test to use it. Don’t get me wrong, I’m not opposed to good certifications, I’m not opposed to teaching and learning. My point is everything you find good in certification have to do with learning, training, and practicing the craft of testing. It has nothing to do with “certification”. The true ugly in this whole discussion is that certifications, in their current state, are self-serving money making schemes that do not advance individuals within the industry nor the industry as a whole.

  • http://www.practitest.com Joel Montvelisky

    Hi Joseph,

    Thanks for bringing forward your point of view on the Certifications Subject.
    I am always happy when more than one points of view are represented in a discussion, and honored when this discussion can be facilitated by my blog.

    There are some comments I have about some of the points you mentioned and I will write them here bellow:

    1. We both agree, as I clearly wrote in the original post, that certifications are not the tool that provides the learning this is done by the courses.
    I don’t agree with your argument that if a tester needs a “formal excuse” to learn as part of this course it makes him a bad tester. I can see how if someone requires a formal course to learn it could make him a bad autodidact, but a bad tester is definitely not someone who cannot learn about testing by himself.
    I will agree that I always recommend people to try to continue learning by themselves, but this is only personal and cannot serve to hand out grades or classify individuals as good or bad professionals.

    2. The certification syllabi of the ISTQB was composed by some of the people who I qualify as top names in the QA & Testing World today, for example Dot Graham, Eric Van Veendendal & Thoma Muller to name a few. Having heard and read the works of these people I personally classify them as QA Professional with much knowledge and experience to learn.
    Are there other professional worth to learn from outside the ISTQB? Absolutely, there are more outside than inside, but this is only a question of numbers, personal affiliations and ideological perspectives.
    My point here is that the syllabus is a great source of knowledge, not a perfect one but one that I feel good teaching and serving as the base to continue expanding.
    Have you reviewed it? It can be downloaded form their site.
    If you have specific points that you don’t agree with the syllabus or that you think that it should expanded you can even suggest these points to the organization.

    2.2 I have heard about the conspiracy theory of the ISTQB, I am part of one of it’s children members the ITCB and I have not seen any evidence or indication that they are abusing or making money illegally or disproportionately. If you have such evidence I would be one of the most interested parties in learning about it.
    Regarding your definition of Scheme, when I checked in my dictionary it read something like: “concise statement or table” or even “systematic or organized configuration”. So I guess their definition is correct on this point.

    (btw, what happened to 3?)

    4. I don’t think the word dictionary is the most appropriate, I would go for glossary. But if you want I can live with dictionary, as long as we are all speaking with the same terminology. And again, if you don’t need the test, good for you!

    Again lastly, if you have evidence of what you call “self-serving money making schemes” I would like to know about them and so would a lot of people I work with.
    If you have it please come forward with it, if you don’t then you might want to choose better words…

    Thanks for sharing your point of view!

  • http://josephours.blogspot.com Joseph Ours

    Thank you for the opportunity to discuss an opposing viewpoint. I think, given the length of the posts and comments, that this may not be the best forum to discuss these ideas. But I’ll try.

    #3 was lost in editing :-) 3. Only once in a while we have the chance to learn theoretical tools that are not 100% linked to our day-to-day work.
    To clarify #3 further, I see your statement as really a statement that theoretical learning is more likely to happen in a classroom, in preparation for a certification test. I disagree. Why is it that people think learning can only occur in a room with 30 chairs and podium? Just today I did some research on Dr. Hawking and his viewpoint on naked singularities. It is not linked to my day to day work, but that doesn’t prevent me from learning about it. What your points 1 & 3 convey is that testers are too busy to learn unless forced into a classroom under the guise of achieving a certification.

    Back to #1. “Bad” tester was perhaps the wrong term. I believe that testing is a thinking activity and not an administrative activity (i.e. not simply running down a checklist). Because of my belief, to me a tester should always be naturally curious, which leads to autonomous discovery and learning. So, if a tester isn’t curious by nature, doesn’t learn unless forced into a classroom, then I would have serious reservations about every hiring that individual as a tester.

    #2.2 – I’m not spouting a conspiracy theory. I do find those theories entertaining :-) , but also disheartening if people believe them on face value. I am not saying the ISTQB is doing anything illegal. The ISTQB is a business, pursuing their own agenda. See their own mission statement:
    It is the ISTQB’s role to support a single, universally accepted, international qualification scheme, aimed at software and system testing professionals, by providing the core syllabi and by setting guidelines for accreditation and examination for national boards

    Let’s following the money first. Consider this. As Michael Bolton pointed out, In Oct 2008, ISTQB announced 100,000 certified testers. Each of these testers had to pay a fee to take the exam. For the U.S, this fee is $250 (entry level) and I think $100 in India. That means they have made between $10 million to $25 million in revenue on certifications alone in the past 5 years. In terms of numbers, they are succeeding at their mission statement. They “sub-let” their exams to training providers who usually want about $2000 USD for a 3-day course. They then provide the ISTQB exam, splitting the exam fee with the ISTQB. These arrangements point to the business that is the ISTQB.

    Tackling their mission statement furthers supports the business premise. They seek to be the sole provider of certification and only ones who can accredit training. Where in their mission statement is any desire to ensure software testers can the most capable individuals they can be or the desire to advance software testing professionals in both theory and practice. It isn’t there. Because this goal is not about advancing testers or the testing industry, it cannot be viewed as anything but a business. They are a money-making business using their “gold star” a.k.a certification as the basis for achieving their revenue goals. ((okay, maybe that could be considered abusive of the term certification for IT professionals – as certification implies a level of skill, capability, and knowledge that the ISTQB can’t currently validate exists in someone who passes the test)) Back to the improper use of certification… This can be seen by one of their training providers recently advertising “The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified.”
    Really, is that all it takes is the stamp of certification? It is completely indefensible.

    On the definition of scheme; let’s set aside the negative connotations of the definition. So, a scheme is “an elaborate and systematic plan of action”. So, through a systematic plan of action they want to support a single certification, where they provide the syllabi and accreditiation. This makes them the king-makers, if you will. And with the king-makers comes the ability to take advantage of that position to make money. So far, they have been fairly successful in achieving this goal.

    Back to #2. I have reviewed the ISTQB syllabus. There is value in that knowledge. By your own words regarding experts in the ISTQB “there are more outside than inside, but this is only a question of numbers, personal affiliations and ideological perspectives.” Two things come to mind. First, if there are more experts outside of the ISTQB than inside, doesn’t that make the syllabus the start of a knowledge repository and not necessarily a great source? Second, if other affiliations and ideological perspectives are excluded from the ISTQB, then it can’t really be about testers or the field, but rather it is just about a single school of thought. And if that is so, then being about a single school of thought with the goals of being the single provider of syllabi and accreditation is more detrimental to the testing industry than simply using “certification” as a money making device. A single school of thought leads to a lack of thinking. A tester who can’t think independently isn’t a tester.

  • http://www.practitest.com Joel Montvelisky

    Hi Joseph,
    I’ve been (and still am) a bit busy taking part of CAST2009 in Colorado, but I wanted to make sure I provided my views on your points.

    First of all, as you see I truly believe in uncensored discussion as a way of, among other things, learn from others and in this effect I really appreciate you taking the time to express your opinions and points of view. This demonstrates that by debating and keeping an open mind we can all learn – as long as we don’t waste time by throwing comments over the wall and then closing our ears to the replies we receive.

    In any case, to the points :)
    - I agree with you that it is sad that many people will only undertake formal education as part of a formal process (e.g. a course, university degree, etc) and applaud the way in which you manage to continue learning by yourself. Whether this happens because they are too busy, to lazy or are simply “not aware” that they can learn by themselves it is something I leave the Academia to determine. I can only speak from my experience, and even this may be influence by my geographical location.

    - I think that curiosity is only one of the attributes of a good tester, an important one but still only one.
    Another one for example is the ability to catch patterns in places most people don’t see them, another one is perseverance, and so on… Not all testers will have 100% of all attributes, this is human nature and diversity, and even if they do they might show it in different ways. I agree that I would prefer a tester who is curious enough to learn by himself, but I know that not 2 people are alike and I also value a lot the rest of the attributes.

    - Regarding your statements against the ISTQB, I can only say the following. I got to meet Michael B yesterday, incredible fellow with great insights into testing! All the rest of the conversation about the demonization or glorification of the ISTQB is already getting me a bit tired, and it was not the intention of this post. I will be happy to take this over coffee when we get to meet eventually in the future.

    - Finally, any syllabus as a source is only the beginning of something, so as you say I see it as a start of a learning path (a sort of kickoff if you like).
    Even the syllabus itself is in constant change and improvement, and I hope that it continues to grow and to incorporate what you call other schools (and I personally hate the word school since I think it is exclusive more than inclusive!), for example I started to see some reference (still too small in mind, but already there) about Exploratory Testing. Logistically speaking I think that it would have been impossible to include a majority of the people when creating the document – who would you have gotten to join the group to add different points of view (and who would be willing to do it and not simply stand up and start screaming how much she/he hates the idea)?. Also, how would you have done it differently?

    As I started saying, I believe in free speech and sharing ideas. I am not sure, as you also said, that this is the most appropriate platform to do this. But at least its better than not doing it all.

    My 2 cents,

  • http://josephours.blogspot.com Joseph Ours

    Thanks Joel. I look forward to spirited discussions regarding the field of testing in the future. As long as we can speak our minds passionately without fear of being ridiculed, silenced, and alienation we can all participate in these important topics to make our industry better. I appreciate your letting me do that. I’m sure we’ll meet up some time some where and I look forward to this and other good discussions.