Taking your product for a successful test drive – How to run a good Beta Program

QuackTest is in the process of sharpening its feathers in order to take its maiden Beta Flight. In fact, there are already a couple of Organizations running the platform in their production projects as part of our initial steps into this vital part of the software development lifecycle…

The Beta is one of the most important parts of the development process. It is the crucial time when the product is initially released to a limited number of customers to be used out of the controlled environment of the testing lab (or developing team environment).

Each Beta is different, in the same way that each product release is different too. For this reason you need to plan each Beta program taking into consideration its individual characteristics, objectives, and constraints.

Do you even need a Beta?

Most products can benefit from a Beta program, but not all of them need it. Home-built applications and specialized custom-built systems will benefit more from a Pilot Project, that is similar but at the same time very different from a Beta Program.

Betas are for products that will be deployed on user environments with multiple and different configurations (more than what we can effectively validate in our Testing Labs), and that will be operated by users on ways that we can seldom anticipate.

The main idea behind the Beta is to perform a “limited” release for users who understand the product has been thoroughly verified but it may still contain unknown defects that need to be found & fixed – it’s the accepted way of releasing your product with a big disclaimer sign, giving you permission to still have bugs…

Defining your Beta

Start by defining the Objectives or Goals you want to achieve. Goals usually fall into 2 categories: Feature validation and Environment (or product) verification.

Since you want to validate specific features and verify a defined set of environments you should look for an appropriate distribution of Beta Users. Even if most times you cannot choose your specific Beta Customers it is important to know what you’re looking for, and maybe perform proactive activities to help you fill in the missing spots.

Once you’ve defined your goals, you will need to review your constraints; specifically the amount of time and resources you can assign to the program.

You will need to provide special support to your users, you’ll also want to quickly process the data you get from the field in order to have a chance to affect the product prior to the release deadline, and you will also be required to handle the issues that seriously affect or harm your users. All these operations take human resources and time to handle.

Finally you will need to understand your time constraints, as for most companies (except maybe google and gmail!) a Beta is limited in time and needs to be completed in order to be able to release the product to the general public.


Running your Beta

It should already be clear from the previous section that the success or failure of the Beta mainly depends on people.

It depends on the users who will run your product and report the issues they encounter; and it also depends on the employees who will gather the issues and take care of the users during the program duration.

Pay attention to both these groups, make sure to prepare them and give them the tools necessary to perform their tasks effectively and efficiently.


Gathering Information

There are 2 ways to gather information during Beta Programs. Most companies only use the reactive way, where they open a communication channel (web-form or email account) for users to report the issues they encounter.

Surprisingly enough, companies don’t create a direct way to gather information from users. A direct channel is created when you interact with your customers and gather first hand information and experiences from their use of the product.

Direct information channels can range from face-to-face meetings to web-based questionnaires where you ask for specific information around defined aspects of the application and user experience.

Running a Beta without gathering all the possible information is like investing a lot of money on a whole set of tools but choosing to use only a small part of them.


Summary

It is important not to release a product to Beta before it is ready, it may harm the confidence of your users in you and your product.

Make sure you have a good team planning, running and handing the beta. Work with your users and help them help you to get the information you require.

As I’ve wrote a couple of times in the past, a fool with a tool is still a fool, and a Beta Program is nothing more than another tool you have at your disposal in order to assure you release the product right.

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

No comments yet.

Leave a Reply