If you are going to write-down just one (more) test case, what would that test be?
Will you write a very descriptive (and deep down) test for the most risky component/screen in your application? Or would you write an end-to-end sanity/smoke test that goes over 80% of your system?
This is a question to ask whenever you sit to write scripted testing scenarios.
Since you don’t have all the time in the world, think about ROI
My guess is that about 95% of the testers reading this blog belong to organizations where time is not a resource to spare. This means you are usually running behind schedule, trying to figure out what tests can be left out and still be able to provide good visibility into the application you are testing.
Don’t feel bad about it, these are the rules of the game. Still, this means that regardless how good your intentions are and how much time you set aside at the beginning of your project, you will NEVER have time to write-down all your deep-down functional test cases.
If this is your reality too, better to be smart about it and write down the test cases that will bring you the best Return on Investment (ROI). Some of the working guidelines I have in place to assure I do this (as much as possible) are the following:
1. Work based on Risks and Priorities
When you have time to write tests, think about your biggest priorities and start with them. It is important not to look only at priorities based on your project timelines, but also based on the risk factors of your application. It may be more important now to write a test for a high-risk feature that will be delivered in one month than for the mid-risk feature you’re getting next week; the reason is that later you may not have a chance to write the test at all. There is no magic formula or excel sheet to define what test to write next, but make sure to keep all factors in mind.
2. Use the 80/20 rule
20% of your tests can cover 80% of your application, this is the principle behind sanity and smoke tests, you can write a relatively short scenario that will uncover a significant part of your bugs.
So here is the answer to question at the beginning of the post: If I can write a deep-down test for the riskier component of my app or a sanity test to go over 80% of the complete system, I will go for the later. I’ll always write the end-to-end sanity suite and only then start covering specific features.
Not only that, when covering a specific feature I will also start by working on the short tests that cover it end-to-end and make sure I have done so for most of my system before moving to go deeper in a single place.
3. Write tests you can give to other people to help you test
A factor in choosing what test to write is to work on the stuff that will help you “outsource” your tasks. For example, write tests you can give to developers to run when they are already done with their features and you are neck-deep in testing high-risk tasks you cannot give to anyone else.
4. Work based on iterations
You wont finish all your writing in one sit, write your tests so that they are “good enough” for now and know you’ll revisit them in the future if you need to improve them. AGILE is not something to be implemented only in development 🙂
5. Think long range and not short range
If you decide to write down a test, make it so that it will still be valid in future sprints/builds/releases; don’t write tests that are too specific and will only be good for the current stage of your project.
Some practical rules of thumb
– Write down the names of your tests before you start writing them. This will help you to get the big picture and prioritize based on the stuff you need/want to do. Don’t worry to much about having a perfect list from the beginning, it will be perfectly OK later on to take one test and break down into 3 or 4, or to take 2 tests and merged them into one. What you want is a list of topics and their risk-based priority.
– Divide and catalog tests into Business Scenarios and Functional Tests, both of them are important but each looks at the system under a different light. The idea of differentiating is to know what test to write and when to write it; and also to help you organize them as part of your Test Library and to help you (and others in your team) in the future to choose what tests can be run based on the needs of your test plan.
– Write tests for your broader audience. Think who may need to run your tests in the future and write them according to their technical level and their knowledge of the system. This will help you to give the test to a developer or even to a fellow tester from a different team and get a helping hand when you need it during your project.
In some occasions it will be impossible to write a single test that suites all audiences and you may consider writing 2 separate versions of a single test.
– Not too long or too short. Tests shouldn’t be too long so that they take half a day to run, nor should they be so short that you need to create a Test Set of 20 tests to cover a single features. Think about tests that between 45 and 90 minutes to run, and that cover a convenient area of the system in a single shoot.
– Don’t write tests you don’t need and run them as much as you can. Writing a good test right the first time is difficult, this means that you may need to run it once or twice while still making changes to them before they are good enough (to be given to others or released to customers). Make sure you test-drive your tests before they go out the door :).
Another important thing is to run your tests regularly or they will stop being relevant. This happens because the application will change, or the environment will be modified, or for a thousand different reasons that will require you to make small changes and to adapt your test to the current constraints. Making small changes is quick and easy, but making full overhauls is hard and may lead you to discard the test and write a new one altogether.
Good test writing is an Art that takes time to learn
and a Craftsmanship that takes perseverance to master.