Release Criteria – defining up-front when the product will be ready to be released

I make a point of not making important decisions in the heat of the moment. For example, when I disagree with someone on a professional level and tones start getting hot during a meeting, I ask for a timeout or a change of subject and to re-take the issue after a 24 hours period of perspective. In 90% of the cases this time will help me and my colleague to better understand the other’s point of view and reach an understanding that allows both of us to walk out of our argument happy.

Still, there are times during a development project when you need to make a decision on-the-spot with no way to “snooze” your assessment. One of these situations happens around the end of the project, during what many of us call Release Decision Meetings; where representatives from all departments get together to provide their inputs and jointly decide whether the product is ready to be released to the public.

I’ve been part in many of these meetings as a QA Manager. These are the times when on the one hand you are pressured to release the product NOW (or as soon as possible), but on the other hand you carry a big part of the responsibility for the stability and the quality of the final product been released.

How do you make sure to be objective and not to make mistakes at these times of pressure? One ways is by having a Release Criteria list that was defined and agreed up front by all the project stakeholders.

What is a Release Criteria? Simply put, it is the list of Objective Prerequisites that should be met by the product and process in order for us to feel confident about releasing it to the public. The most important aspect of the Release Criteria is that it needs to be reviewed and agreed by all product stakeholders prior to the Release Meetings and preferably during the early stages of the Development Processes.

The release criteria will never replace the Release Decision Forum, as this group of people needs to take into account all objective and subjective aspects surrounding this decision; but it will help you (the QA Manager) to provide an Objective Evaluation based on the parameters that were considered important before we started being influenced by external and maybe inappropriate factors.

What comprises a good Release Criteria list will vary from product to product and from company to company, but some basic items that should be included in it are:
1. Percentage of the original required functionality that needs to be in final the product.
2. Testing coverage level – percentage of the tests that need to be run on the application from the original testing goal.
3. Pass rate for the testing coverage –percentage of the executed tests that need to be successfully run without falling into blocks or failures.
4. Number of critical defects still open on the product.
5. Maximum defect detection rate for the last month, week, or x amount of days– based on the assumption that enough testing is still being done on the product.
6. Minimal load tolerance (if applicable).
7. Minimal amount of time the application has been running continuously on the testing and/or staging environments (if applicable).
8. Minimal amount of time the application has been on a Beta Testing trail (if applicable).

It is always recommended to start updating and publishing the status of the project with regards to the Release Criteria as early as logically possible. If you have Kitchen Monitors in place you should consider adding a page with it the to your running template.

As with all information generated by the QA, using the Release Criteria wisely will increase the amount of value you provide into the process.

One Response to Release Criteria – defining up-front when the product will be ready to be released

  1. Harshal February 29, 2012 at 8:24 am #

    How to form Release criteria for productA when there is another productB aligned with it and having different release schedule?

Leave a Reply

Shares