Smoke & Mirrors…? – What exactly do we gain from Process Certifications?

I ran into someone looking for information about the CMMI. He was interested in the criteria that placed a company (or better defined, its process) under Level 3 instead of Level 2.
This reminded me of past times when I took part (and even lead) the certification processes in similar Organizations.

I don’t intend to demonize Certifications. I don’t think they are bad for the Company or its customers, and I certainly don’t claim to be smarter than the people who run the CMMI or ISO certification process or to have an alternative to what they are trying to achieve.
I just want to share with you my point of view about the meaning of these certifications and what a potential user can learn from them.

Who gets certified and why?

I know of 2 types of companies who get certified, the ones who do it since they think it is something that will improve their processes and practices, and those who do it since a specific market or competitive condition requires it (i.e. CMMI for the US-DoD, CFR 21 – part 11 for the FDA, ISO to compete in the EU, etc).

As you may imagine, the vast majority of the companies I met fall into the second category.

A thing that surprised me at first was that there is no relation to the size of the Company and it been certified (and why should there be?!). I was always under the impression that only big companies sought certification, but then I learned that many small firms take advantage of this common misperception and get certified in order to appear more robust (and at times larger) than they really are.

How do you get certified?

The process is very straight forward.

All certification bodies (CMMI, ISO, FDA, etc) have a set of publicly available documentation that explains how to comply with their criteria. These documents are basically a (very large) set of checklist with explanations, definitions and even examples about each points that is evaluated during the appraisal process.

A company seeking certification will either get an External Consultant or someone in-house to make a Gap Analysis showing what’s already available in the process and what areas still need to be defined and/or improved. They then create a task team to define and implement the missing processes.
When the new processes are implemented a follow-up analysis or Appraisal Dry Run is performed to make sure that the Company is ready, and if all goes well it requests a formal appraisal to be made.

The appraisal itself is usually done by a group of certified auditors that come to the premises of the Company for a couple of days and perform random checks on the process and specially the documentation that accompanies this process.

They look for a logical and continuous process that is correctly documented and followed in the different groups and products of the company. Based on my experience they look for traceability and usually like to perform wide reviews (take one product and make a complete run from start-to-end) instead of deep ones (take one part of the process and make sure all products have the same docs), but I guess that this can change between auditors and certifications.

At the end of the audit the Company gets an Appraisal Report. It contains results and comments for each part of the process, and it almost always includes feedback with improvement suggestions or requests.

The Company is not required to do PERFECT and thus if it has non-critical issues it will still get the certification, although it will be required to show improvements for these issues in follow-up audits. If the auditors DO FIND things that are critical, they will provide this information and suggest improvements to be done. After their implementation the Company will need to undergo the whole process once again.

Most certifications require the Company to undergo one scheduled audit and a number of un-scheduled audits a year. But this changes between each certification body.

What do we gain from the Certification?

Using my typical sarcasm, the biggest added value of the certification is the Certification itself. The Company will be able to brag about it or show it to a potential customer who will be able to mark-off that specific check-box on their selection criteria.

Taking my sarcasm mask off; a Company working under a defined and thought-off process will be able to waste less time juggling and trying to re-invent answers for common process-related issues.
As an Organization grows and starts having multiple independent groups working in parallel it becomes harder to look for synergies or even simple re-uses of knowledge, experience or code; this can be solved by having a standard process and information systems that everyone knows and uses.

There are obviously more advantages to a certified process; such as repeatability, forecastability, visibility and control; but since they are fairly trivial I won’t review them further here.

What are the problems with Certification?

The main problems are those related to the misunderstanding of the customers and the way many Companies take advantage of this.

The Certification doesn’t mean about the product; it doesn’t say anything about its correctness or even its quality. You can develop a product that doesn’t answer the requirements and is full of bugs, but as long as the documentation is in place and the process is followed your Company may still get the certification stamp. Most times users don’t understand that they still need to validate the product and test it before they buy it.

Additionally, certifications mean a lot of paperwork and red-tape (even if today it is Virtual Paper and Information-System-based Red-Tape). This means that a Company seeking to be CMMI or FDA certified will need to “pay a penalty” in flexibility and its ability to juggle solutions quickly out-the-door.

My personal take is that Certifications are a necessary evil :o)
In principle they are good and they make us work in a more serious way, but they come with a high overhead that needs to be understood before jumping into it.

If a Company is looking for process improvements they don’t need to get certified, if there is enough management support they will be able to achieve all the added value of the certification with a lot less pain and red-tape by defining a Process Leader in-house.

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

No comments yet.

Leave a Reply