One of the most important criteria (if not the most important) is how much to spend, especially since the range of solutions today starts at $0 for freeware products and goes all the way to $5,000 per seat for some of the Enterprise Platforms.
Before moving forward I want to provide some insights around the freeware solutions. It is true that they are available based on a free license agreement, but take into account that these solutions do not always come for free; there are “hidden costs” you should take into account, for example:
– Hardware & Software for the installation platform
– IT or consulting time needed to install and configure the system
– Migration and customization costs to make sure your data and process are transferred into the tool
– Additional add-ins or plug-ins required to integrate the system to other tools or systems in your working environment
These costs can be as little as a couple of hour’s work of one of your testers and/or IT guys, or as much as thousands of dollars from a consultant who’ll do the job for you.
Once the question of price has been put aside you should concentrate on the functionality and value you are looking for. You’ll need to create a list of things you need from your tool, and for each requirement understand how much effort will it take to achieve it with each of the alternative tools.
An initial list of functionality to look for may be:
– Flow customization (states and transitions)
– Fields and form customizations
– Graphs & reports (static & statistical)
– Ability to easily import and export data from and to the system
– Mail notifications
– Integration with Configuration Management systems
– Integration with Test Management tools
– Integration with CRM platforms
– Permission management
– Accessibility and good response times from geographically dispersed users
When going over each requirement take into account that the evaluation will seldom provide an answer of Yes or No. Most tools claim that the desired functionality is achievable via some sort of API or Customization Module, if this is the case you will need to estimate the amount of work required to achieve each of the features.
MAINTENANCE COSTS & LIMITATIONS
Last but not least, evaluate the on-going cost and limitation you will be getting as part of your tool. This list is almost never published by the vendor or developer and you will need to do your homework thoroughly (or consult with someone who has experience with the tool) in order to reach this information.
The types of things you want to include in this list of burdens are:
– Supportability – how easy it is to get help in case something is not working correctly.
– Scalability of the solution
– Ease of maintenance operations such as upgrades, back-ups, migrations, etc
Most of the inputs required in order to get a good understanding of the tool come from experience or from talking to existing users. Look for people from your geographical area and/or who share the same type of requirements you have from the tool and consult with them. If you still have no way of finding such a person, download the tool and run a pilot project with a small amount of data and employing as little time as possible in order to reach the necessary conclusions.
The selection criteria is critical in making sure that you and your team will work with a tool that helps to streamline your process and not one that stands on your way and lowers your efficiency. Take the time and care necessary in choosing the correct solution for your needs.
Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.
Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.
With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.