When we test, the idea is not to find all the issues, we only need to focus on those issues that will affect our users.
The question of “If a tree falls on an empty forest, does it make a sound?” is simple in the context of testing… If there is a bug that no one will run into, then we do not care about this bug!
But life is never simple, and so the question usually is:
How do we know if the issue will be an important to our users???
Many years ago this was a very hard question to answer.
15 years ago we used to ship products in CDs that were installed in desktops or on isolated servers, then it was very hard to get direct feedback from users.
We would get lists of bugs from our support teams, Product Managers would talk about feature requests, and if we were really lucky we would hear from our Professional Services teams about how people were deploying our systems.
Back in those days we would make assumptions about the usage of our system, generate test cases, invent Personas, perform triage analysis when we would not agree on the priority of bugs, and overall pray for the best results to come 🙂
Today most of us can actually see how our end users work with our systems.
We’ve undergone two very important evolutions that changed this reality completely.
The first is a technical one – Cloud Computing.
Today many of us are able to release our products the cloud instead of shipping them to our users. This means that we are also able to monitor the way end users actually work with our systems.
This is not automatic or trivial, but if your company works correctly and you thought about this requirement while designing the product, you will have instrumentation to track what actions your users are doing, where they are working (and where they are not working), if they run into any exceptions, errors or roadblocks.
It is almost as if you could get a chair to sit behind your users in order to see how they are working with your system. Something we used to do in the past by visiting them on-site or by having expensive usability labs in our companies.
The second is methodological – DevOps.
Once we started working in the cloud and deploying our applications in our own server farms it became obvious that the best people to deploy and manage the systems in production where the same people who were writing these systems.
The only problem was that developers lacked the knowledge of how to manage operations’ systems in large scales and with multiple servers and services interacting in parallel.
Long story short… We merged our Agile Development Teams together with our Cloud Operations Teams, and we got the new concept of DevOps.
DevOps also meant that, together with our Agile Development Teams merging with our Ops Teams, we also injected testing specialists into these teams, and so we can start merging our testing operations before and after the deployment “event” takes place.
You can stop making assumptions about what your users are doing, simply look and learn
What has all this to do with users and profiles, hopefully you understand by now that it has everything to do with it.
If once we had to make assumptions regarding usage patterns and what functions were most frequently used in the system, now we can simply query our monitoring systems to get this information.
If you have a real user database, you can now generate Personas for your team that are more aligned with the reality of your users.
And maybe most importantly, if once we could only measure the quality of a release based on the escaping defects that reached our support team 3 or 6 months after the date of release. We can not get real time feedback about issues in the system, and most times solve these issues within minutes of the release “event”.
What’s even better, we can add a new dimension to quality of the system! As we can now see if users are working with the features we wanted them to work with.
For example, if your team just worked for months on a new reporting module, but users do not use it, then you know you have a problem. It can be a UX problem, or a feature definition problem, or something else completely. But fact of the matter is that you invested a lot of efforts and they are not paying out in the way wanted it to. So you have a quality issue.
Years ago we would have needed to wait months to get feedback on features. Today we can get this feedback in hours or days.
There are many other positive things that came out of DevOps and the Cloud…
There is shifting-right or testing in production, there is the fact that you can get environments that are similar or identical to the production environment to test, and many many more things.
But we will leave those for future blog posts 🙂
Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.
Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.
With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.