PractiTest QA Blog

For those who are not up to date with the latest buzzwords SaaS stands for Software as a Service; it means that the software platform is hosted and managed by the vendor and users simply open their browsers and connect to the application from anywhere in the World.

Part of my daily activities include being in charge of the QA and Testing for PractiTest, a SaaS QA Management System used by QA and Testing Teams to manage their Test, Bugs and Process.

One of our customers asked me how is testing a SaaS platform different from testing any other project or product?
In principle I don’t think it is different, but it requires us to merge various types of tests that up to now I’ve only used on isolated projects.

To give a better explanation, here is a very quick list of the types of tests we constantly run on PractiTest:

Functional Scenarios – As with any other application we make sure that the functionality works as expected.
For this we use the regular testing techniques such as:
- Manual test scripts
- Exploratory charters
- Checklists
- End-to-end scenarios
etc
Apart from manual testing we are also starting to *play* with selenium in order to create an automated regressions testing suit.

Multi-platform support – We use a combination of physical and virtual machines to make sure PractiTest users can work on Windows, Mac and Linux; an also on the IE, Firefox, Chrome and Safari Browsers

Load and Stress – PractiTest needs to handle large amounts of users and we don’t have the luxury of re-booting or going down once in a while.

Remote Access and Usage – Simply put, we make sure that all users regardless if they come from the US, Holland, India, Argentina, or Australia can work with the system with good response times.
There are many emulators (such as Shunra) that can help test this in your lab, but there’s nothing like the real thing, and for this we have a number of local collaborators that are willing to perform a set of tests and report their results from their locations.

Security – We make sure no one can harm (consciously or by mistake) their system or the platform. For this we test that things like cross-site scripting and other security loop-holes are blocked.

Live updates and deployments – Here is something we seldom think about in regular applications: How do you deploy the system while it is still running, and if needed how do you minimize down-time for the users?
This is something that is developed and handled by our IT department, but since the delivery and update of the product is part of the overall user experience we test it constantly.

Disaster recovery & rollback procedures – Another testing task coming to us from the IT side of the house.
Here we run 2 main scenarios:
1. System down that needs to be brought up quickly (with the same machines or with new ones that require installation and configuration)
and
2. Rollbacks to the last known stable version, including data.

I18N – Since our platform is used by people around the world we make sure that we support the use International Characters.

There are additional tests and activities we are constantly adding to our testing suites; but the main principle is that we’ve learned to see our AUT (application under test) as the complete end-to-end user experience and not only the functional application where testers (our users!) do their work.

We understood that we don’t test only to make sure the functionality works. Our testing objective is that users get the best possible APPLICATION AND SERVICE from our company.

Last week I wrote about the Project Guy Ariely from AQUA was organizing, where people from the Testing Community in Israel donated their time and experience, and gave short presentations (90 min to 3 hours) about various topics related to Testing and QA.

I presented a session about QA Management Systems, and based on the questions asked there were in the room at least 10 to 15 Testers I’d like to interview next time I have an open position in my QA Team.

Where’s the problem, and I’ve seen this in Israel but I am guessing it is also an issue in other places.
Many testers looking for a job today do not have a College Degree in Computer Science and this makes their CV less valuable in the eyes of the average QA Team Leader and Manager. This problem becomes even more acute as a large number unemployed testers today have this degree as a line in their resume.

So here is my request to all the Team Leaders and Managers looking for testers:
STOP MAKING A CS Degree A REQUIREMENTS FOR A TESTER!!!

When are we going to finally realize that a Tester is not a type of Developer?

A tester needs to understand about programming, but we don’t need a formal degree on it (BTW, some of the most incredible programmers I know don’t hold a CS degree either!).

Instead of in-depth knowledge in programming languages, look for people who can understand the human language and can translate user requirements into tests that verify their real and complete needs.
Instead of looking for candidates with knowledge in development algorithms, look for people who are curious and organized by nature and don’t need an existing path to do their job efficiently.
Instead of someone who can inspect one line of code for hours at a time looking for a way to fix a bug, find the testers who will think “outside the box” and in a minute come up with 7 different scenarios to run on the new feature (and find the unimaginable bug!).

In short, understand what you look for in the person you want to hire, and then decide whether he needs a CS degree…

To the people without a CS degree looking for a Testing Job today (in Israel and maybe elsewhere), first of all understand that you are at an unfair disadvantage. Better to know this than to not understand the reason you are not getting as many interviews as the next guy.

Now, think about what you can do to compensate for this in your CV. I don’t have a magic formula, but you need to think about ways of getting around this issue and making your resume “look good” even without such a degree. One way of trying to do this is based on your personal and professional experience…

Lastly, realize that most of the good jobs are seldom published on Hiring Bulletin Boards, they are filled based on personal recommendations. So make sure all your friends, family and neighbors know that you are looking for a Job and are ready to recommend you next time someone asks for a good tester during an informal chat.

It’s in your hands to catch that Job before someone else does…

A closing comment:
If back when I started testing a CS degree would have been a prerequisite, I would not be writing this post since I have a BSc in Industrial Engineering…

Times are tough, but in some cases this helps to bring out the best in some people.

Like a colleague here in Israel, Guy Arieli from AQUA, who came up with an idea to help Test Engineers looking for a Job who want to expand their theoretical knowledge in Testing and QA.
He organized a FREE 8 session course where the Engineers learn about stuff ranging from Freeware Automation Tools to the Basic of Mobile Application Testing, and cover important practical topics like preparing for Job Interview and the like (this is the site for the course in Hebrew).

Each session is imparted by a different person from the Industry, providing a lecture on his topic of expertise; and the lectures themselves take place in HP R&D’s Campus in Yehud.

This evening I will be talking about QAMS (QA Management Systems), hoping to communicate the importance and advantages of using this platform as an engine for Testing Intelligence. Obviously this is not much, but I hope it helps.

Kudos to Guy for the great initiative.

My wife gave birth to a healthy baby girl last Thursday at 6:18 am.

Before I write another line I feel the need to say something to the women readers of these posts:
WOW!!!
There’s a reason God put you in charge of delivering babies, we men simply don’t have the guts to do it as well as you do!

Since this was my second time as a Dad on the Delivery Room, I was able to pay more attention at the things going on around me (instead of worrying like crazy and standing in the way of all the people trying to do their jobs…)

I took special care to understand what the Delivery Nurse (who does most of the work there) did before, during and after the delivery itself.

My biggest surprise came when, after the birth was done and the nurse was documenting all the important stuff in the computer, I saw her typing down numbers under a section titled “APGAR Test Results”.

Being a Tester I had to understand what the numbers meant.
So, very discretely I approached her and asked for more details on the meaning of the Tests and their results. To tell the truth I didn’t understand what she did, but after I got home and slept for a couple of hours I turned to good-old Wikipedia.

In short the APGAR is very quick test performed by the Nurse or Doctor one minute and five minutes after the birth itself, they go over the 5 more basic and important things (skin color, pulse rate, reflexes, muscle tone, and breathing) and serve as a first indication of the general health of the baby.

This sounds pretty much like a kind of Smoke-Acceptance Test run on the Baby just after delivery.
They even have clearly defined pass criteria and fail criteria.

Who would have thought that even a project such as Pregnancy and Birth would have an Acceptance Test at its completion… )