9 Cognitive biases and how they influence software testing

9 Cognitive biases

We make hundreds of decisions, choices, and judgments on a day-to-day basis. Life would be exhaustive if we had to deliberate over each and every one of these decisions. Humans prefer to avoid information overload by preferring solutions that require minimal mental effort. We are naturally programmed to handle information overload by using mental shortcuts called Heuristics. Heuristics enable quick decisions over and over again using past experiences. Heuristics are fallible and lead to cognitive bias, stereotypes, and prejudice. Cognitive biases are systematic subconscious errors that hinder our rational decision-making when processing and interpreting information.

Most people consider themselves as rational beings. But we are not as rational as we think we are. Human behavior largely relies on intuitions, emotions, faith, feelings, and primitive instincts that were designed to survive. There are around 188+ recognized biases. Cognitive biases impact our day-to-day work and hamper our critical and rational thinking leading to erroneous decisions. Similarly, when performing software testing, we fall prey to some unconscious psychological biases. Let us look at some of the dangerous biases that can influence and affect software testers and how we can overcome them.

 

  1. AVAILABILITY BIAS 

The availability bias is the tendency to make judgments about the likelihood of an event based on the readily available examples in our mind. For instance, we might assume that the most number of deaths caused this year in the United States is due to COVID-19. This is because we see many stories in the news about COVID-19. We may judge that the likelihood of deaths due to COVID-19 is greater. The fact is that heart diseases have caused more deaths in the US than COVID-19 this year.

media
Media coverage can fuel a person’s availability bias 

 

HOW AVAILABILITY BIAS REFLECTS IN SOFTWARE TESTING? 

The development team and the management feels that there are a lot of issues showing up in production by recalling two recent examples of production issues. They may judge the tester and feel that his/her work is not up to the mark. The fact is that the tester never leaked a bug to production in the last 6 months and always did a great job. There were only a couple of instances when the bugs were leaked.  The development team and management were the victims of availability bias. 

Every tester needs to be data-driven. Establishing and tracking quality metrics is essential to combat availability bias, When people have biased opinions, testers should present relevant metrics to eradicate such biases.

PractiTest Dashboard
PractiTest Dashboard

 

  1. REPRESENTATIVENESS BIAS 

Who is most likely to be a wrestler? Jon or Jim? 

Jon or Jim

Most of us would predict that Jon is likely to be a wrestler. Because Jon perfectly fits our stereotype of a wrestler. The fact is that Jon is a musician and Jim is a wrestler. The representativeness bias is the tendency where people see the commonality between people or objects of similar appearance and make incorrect stereotypical assumptions. Basically making judgments based on representativeness.

 

HOW REPRESENTATIVENESS BIAS REFLECTS IN SOFTWARE TESTING? 

In spite of the advances in technology and in the age of DevOps and Agile, a lot of negative stereotypes and misconceptions about the testing still persist like –

  • Testers cannot code. Testers cannot understand technical stuff and underlying technology.
  • A failed developer is pushed to testing by the management as they feel testing is easy and one can test.
  • Quality is the sole  responsibility of the QA team
  • Testers are gatekeepers of quality. 
  • Automation will kill the jobs of testers one day.
  • And a lot more….

Many testers feel like they lack the respect of their peers due to some of these reasons. Many of these stereotypes are present because Management and developers do not understand testing. It’s important to build credibility as a tester to grow in your profession. Firstly value yourself and your profession. Then back up your opinions with data, educate your team about testing, develop a technical mindset, upgrade your skills to break the negative stereotypes around testing.

 

  1. ANCHORING BIAS 

Marketers are able to capitalize on a cognitive bias called anchoring bias and influence a customer’s reasoning ability. We have the tendency to rely heavily on the first piece of information that is offered to us. The first piece of information acts as an anchor and influences our decision making. 

original price anchor

Here’s an example from an e-commerce site where the original price is used as an anchor to influence your decision making and consider the discounted price as a great bargain.

pricing plans

Some SaaS products list their most expensive plan first which acts as an anchor and makes other plans seem more affordable and attractive in comparison, influencing your decision making. 

 

HOW ANCHORING BIAS REFLECTS IN SOFTWARE TESTING? 

Anchoring bias can affect your testing estimates. 

Jim works as a tester in an agile team. In a sprint planning meeting, the product owner asks – “Hey Jim, How much time do you think it takes time to test this new feature?” The product owner also adds “I think it should not take more than 8 hours. What do you think Jim?”. In this case, the first piece of information(8hrs) that the product owner offered acts as an anchor and can influence Jim’s estimate. Jim might end up giving an estimate which is close to 8 hours without being rational. This may later lead to inaccurate estimates and delays in releases. Teams should adopt appropriate agile estimation techniques like planning poker is to avoid the influence of the other participants in providing estimates.

When making complex decisions, do not act hastily, slow down, and get additional information instead of making a quick decision under pressure. 

 

  1. GAMBLER’S FALLACY 

GAMBLER’S FALLACY 

The gambler’s fallacy is an erroneous belief that the chances of an event occurring with a fixed probability become higher or lower if the process is repeated.  The probability of a coin turning up Tails is always 50 percent, the same applies for Heads. Each flip is independent and it does not matter what happened in the previous flips. But we have a tendency of getting influenced by past flips. If I get heads back to back thrice, I may predict that the next flip will result in a tails by getting influenced by the past flips(assuming that the probability of the tails showing up will be lower). This bias has led gamblers to lose millions of dollars over the years. In software testing, this bias can make us look at patterns that are not existent and lead to errors in judgment. 

HOW GAMBLER’S FALLACY REFLECTS IN SOFTWARE TESTING? 

An agile team felt that there is a connection between the number of releases and issues reported by customers. The team felt that more releases lead to more issues. The metrics they collected also supported this claim. So the team planned to slow down on the number of releases they make assuming that customer-reported bugs will reduce. However, the bug count did not go down in spite of lesser releases. Later, upon detailed investigation, the team realized that customers mostly reported legacy issues that had nothing to do with the new releases that the team was making. Noticing patterns is a natural tendency for humans, so this bias can subconsciously lead you to make costly mistakes. To avoid such biases ensure you drill deeper into the data. Look at metrics, numbers, and trends from different angles. Look at the same data in a variety of ways so that you are not a victim of gambler’s fallacy.

  

  1. PREMATURE CLOSURE

The tendency to stop too early in an investigation process and arriving at an early conclusion before gathering all the required information is called Premature closure.  It’s basically the belief that we have arrived at the correct conclusion and hence prevent further verification. This is one of the most dangerous biases that testers need to look out for. 

HOW PREMATURE CLOSURE REFLECTS IN SOFTWARE TESTING? 

The set password field requires the user to add a special charter mandatorily. as shown in the picture below 

A tester when testing this feature he notices that the ‘set password’ field does not accept @ and # characters. The tester comes to an early conclusion that the ‘set password’ does not accept any special characters and is a blocker for signing up. The tester reports this as a severe issue. But when the developer tests with a special character “!”, the signup works fine! Then the developer then marks the issue as Invalid. 

In this case, the set password fails to accept only a subset of all special characters. But the tester was a victim of “Premature closure”. This is a strong bias that can damage the credibility of the tester. Before reporting bugs verify if you are able to replicate the bug multiple times by varying test data.

To prevent premature closure, retrospect on your decision making and think about instances where you rushed to wrong judgments by arriving at early decisions before the complete investigation in the past. 

  1. THE BANDWAGON EFFECT

The bandwagon effect is a psychological phenomenon in which people tend to do something fundamentally because other people are doing it, often neglecting their personal preference or underlying evidence.  We naturally tend to avoid complex information processing and decision making by relying on other actions.

bandwidth bias

 

HOW BANDWAGON EFFECT REFLECTS IN SOFTWARE TESTING? 

  • Implementing Lean/Agile/DevOps practices since everyone seems to be doing it.
  • Deciding a career path getting influenced by the majority.
  • Many meetings may turn out to be unproductive due to this bias because team members don’t challenge collective agreement and confirm to it.

This bias can also impact your decision making when evaluating testing tools. 

  • When kick-starting an automation initiative, QA leaders may consider only the open-source tools like Selenium simply because it’s popular and everyone else they know is using it, without analyzing whether or not it satisfies their use cases.
  • Using spreadsheets for test case management, only because other teams are doing it. Spreadsheets are not meant for test case management and as teams scale, spreadsheets get lost and cannot ensure visibility, traceability, test case reusability, etc which an advanced test case management can offer. 

To avoid the bandwagon effect, do not jump to conclusions without validating the required information. Consider other alternatives that are not backed by the majority. Go with the majority only when you have enough factual evidence. When making important decisions separate yourself from others to decrease their influence on you.

 

  1. SIMILARITY BIAS

SIMILARITY BIAS

We tend to surround ourselves with people who are similar to us and with whom we feel naturally comfortable. We enjoy working with people who are similar to us.  This is called the similarity bias. 

HOW SIMILARITY BIAS REFLECTS IN SOFTWARE TESTING? 

  • Hiring managers tend to hire candidates in whom they see a part of themselves. The candidates who think, look, behave, and are culturally aligned to them. The hiring manager may end up building a team with fewer perspectives and ideas. Diversity is the mother of innovation. A diverse team has more options, innovations, perspectives, and growth opportunities.
  • Similarity bias can sabotage performance reviews. Similarity bias can influence QA leaders to judge and rate a similar subordinate more favorably and have a positive relationship with such subordinates. Similarity biases may end up in making errors in crucial decisions like who to promote, who to assign to critical projects.

Hiring managers should practice appreciating and objectively assess candidates who think differently from them. Embrace a collaborative hiring process with a diverse hiring panel to eliminate similarity bias. 

 

  1. CONFIRMATION BIAS

CONFIRMATION BIAS

Confirmation bias is the tendency to look for evidence unconsciously that confirms your preconceptions and opinions. You ignore or dismiss the facts that disagree with your preformed opinion. 

HOW CONFIRMATION BIAS REFLECTS IN SOFTWARE TESTING? 

  • Peter is a tester and works with two developers Jim and Jon. Peter strongly feels that Jon is a terrible developer and his work is never up to the mark. He feels that Jim is a better developer than Jon. Peter sub-consciously finds evidence to back up his opinion and tests Jon’s code very thoroughly to find several issues, however, in the process, he may ignore good parts of Jon’s work and also focus less on Jim’s work leading to issues in Jim’s modules. 
  • Testers tend to search for information from user stories or requirement documents that confirms and reinforces their existing belief and knowledge about the software under test.
  • If a tester does not like the automated tool/solution that is chosen, he/she tends to look for more gaps and cons in the tool/solution to justify his opinions.

In software testing, confirmation bias manifests as “Positive test BIas” which is the tendency to execute as many as positive tests to demonstrate that the software works as expected.

  • Favoring the selection of regression test cases in a test run that testers believe will execute according to his expectations. 
  • When automating, testers tend to choose the test cases and data and that will work as expected to ensure that their automated tests run without hiccups – reducing the bug detection rates from test automation.

To combat confirmation bias, the most important step is to acknowledge that confirmation bias exists. Additionally, clear your suspicions by creating multiple hypotheses and seeking out information from different sources that go against your pre-existing notions.  

  1. AUTHORITATIVE BIAS

 AUTHORITATIVE BIAS

The tendency to appeal to authority and attribute greater weight to the opinion or thoughts of authority is called Authoritative bias. We tend to get influenced and count on the information coming from an authority figure, regardless of the accuracy of the information. This is the reason why we see celebrities and experts are used in advertisements by marketers.

HOW AUTHORITATIVE BIAS REFLECTS IN SOFTWARE TESTING? 

The Engineering manager asks the tester to skip testing a feature to production due to time crunch and informs the tester that the developers have done the testing themselves and nothing can go wrong. The tester agrees with the engineering manager without a second thought. Post-release a critical bug is found in that feature that impacted multiple customers.

One of the most important attributes of a tester is to ask questions. Testers need to ask the right questions to their leaders instead of blindly following their manager’s thoughts or opinions. This bias is not about dismissing claims, thoughts, or opinions of the more experienced and knowledgeable. Following the ideas of authority has great benefits as you don’t have to spend additional time and energy figuring out things for yourself. Trust your authorities but verify and validate their opinions and thoughts.

 

CONCLUSION

For most of human history, our survival was primarily dependent on our ability to make quick decisions when we did not have time to think of all possible logical solutions. 

conclusion

 

We are all biased and we cannot avoid it to a large extent. However, we can manage our biases. The first step towards overcoming these biases is to acknowledge that we have them. Awareness is the key to battle biases. Being aware of various cognitive biases enables you to counter their influence over you, and lead to effective and unbiased decision making. 

Let data drive your decisions instead of guess and intuition for critical decisions. Diversity can also help testers to combat some of these biases. Interact with people who will challenge your opinions and thoughts. Hear other’s thoughts and validate even if it contradicts yours. Use approaches such as the Six Thinking Hats, Five W’s technique, mind-mapping, SWOT analysis, etc to validate your decisions from multiple perspectives. Practices like Pair Testing, Brainstorming, Code reviews, Peer reviews, Mob-testing, Bug-bashes, etc can give you different perspectives and ensure the rationality of your decisions and testing activities. 

 

Image credits

 


About author

Prashant Hegde

Prashant Hegde is a passionate tester and an agile enthusiast. He’s led test teams to success in many organizations, helping them improve their application quality process. Prashant currently heads the QA team at MoEngage

Prashant is a frequent speaker at industry conferences and enjoys sharing his experiences by blogging and participating in agile communities around the world.

Prashant has helped thousands of ISTQB aspirants to clear their certifications with his free app, ISTQB Test Mentor.

About PractiTest

Practitest is an end-to-end test management tool, that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

, ,

No comments yet.

Leave a Reply

shares