Avoiding Bias in Market Research: Types of Bias

Bias happens. Just being human means you have many biases. They’re inherent in our preferences, our likes and dislikes, our expectations and experiences. Biases are underlying factors that cause prejudices that can lead us to favor one thing over another, even though the thing in question may have little or no material difference.

In a market research context, bias is any error in the research processes that skews the results in a particular direction. It can be stealthy. You may not be aware that bias has crept into your research efforts until it’s too late.

Research that is influenced by bias does not reflect the state of variables that exist in the real world. For example, a study on consumer preferences for laptop configurations that surveys only people over 25 will certainly not reflect the entire addressable market. Once bias has infected your research, it can lead to bad decisions that can be both dangerous and expensive.

Common Types of Bias

There are many ways bias can creep into your research efforts, so it’s important to develop a strategic plan to keep it from invalidating your work. This might mean using mixed methodology research (i.e., a combination of qualitative and quantitative research studies) to gain reliable insights that you can trust. But different types of research require different strategies. Understanding the different types of bias in research is vital to executing a methodologically sound study.

Confirmation Bias — Confirmation bias refers to the tendency of researchers to interpret information (e.g., data, charts, or ambiguous signals) in a way that confirms their beliefs. The danger here is that any information inconsistent with your beliefs is given less value (or rejected) than information that is consistent with your beliefs. The insights that will emerge from research colored by this bias will not reflect reality.

Avoiding Confirmation Bias — Awareness of confirmation bias is your best defense to ensure you gather and weigh information from both sides of an issue. Be conscious and thoughtful about research design. It can be advantageous to employ researchers who don’t have a direct interest in the results of the research and therefore can design the research, execute the study, and analyze the results without perpetuating any confirmation bias that might otherwise exist.

Acquiescence Bias — Human beings tend to have a psychological need to be perceived positively by the world. As a result, they lean toward “agreeing” or “strongly agreeing” with certain research questions. For example, if a survey on consumer satisfaction asks about a respondent’s satisfaction with a particular product, they may respond more positively than they believe. People don’t want to be perceived as a “bad person” and will answer in a way they perceive to be most socially acceptable/safe. Acquiescence bias can also occur when the survey is poorly targeted, or the respondent is not engaged enough to answer thoughtfully.

Avoiding Acquiescence Bias — This bias is important to understand both when designing and interpreting research. Design-wise, if you ask a respondent to rank their agreement with a question like “I am satisfied with my current cell phone,” they may be more likely to respond positively. Rephrasing the question as “How would you rank your satisfaction with your current cell phone?” can generate more authentic responses. After the research is complete, it’s necessary to evaluate and balance the results while considering how a respondent’s background and education might have influenced their responses.

Sponsor Bias — Sometimes research can be biased when the respondent knows who is conducting it. A respondent who knows that the research is being conducted by Company X could be influenced to answer more or less favorably.

Avoiding Sponsor Bias — To avoid sponsor bias, the best strategy is to “blind” respondents from who is conducting the research. When respondents don’t know who the research is for, they are more likely to give their true opinions. In these cases, it’s often best to use a neutral third party to conduct the research on behalf of the sponsor.

Selection Bias — Selection bias can occur when the sample does not accurately represent the population of interest. For example, if we want to understand consumers’ perception of pricing for a new smartwatch, but we sample only people who are in-store shoppers, we may be losing the insight of online shoppers (who may be willing to spend more than in-store shoppers, or vice versa). This results in a skewed understanding of the market.

Avoiding Selection Bias — The ideal sampling method would be using a completely random sample. But that’s not always possible. Instead, researchers should strive to use representative sampling (proportionate to the population of interest) by stratifying the sample across different segments (e.g., geography, organization size, etc.).

In cases where selection bias is hard to avoid (i.e., limited populations and limited willing respondents), it is especially important (and even more so for qualitative research) that potential research respondents are properly qualified and vetted, and that what they share is viewed more directionally.

Biased Questions — While this isn’t necessarily a type of bias, how you serve questions to respondents can potentially bias how the respondent answers.

• Double-barreled question — A question that addresses multiple topics but allows for only one answer (e.g., How satisfied are you with your cell phone and cell phone provider?). Double-barreled questions can confuse respondents, obscure the true meaning of the question, and lead to inaccurate data.
• Leading Question — A question phrased in a way that guides a respondent to answer in a certain direction (e.g., Many people find the current website easy to navigate. What do you think of the site?). The direction is often favorable for the research sponsor and/or validates confirmation bias.

Questions like these can prevent a respondent from providing authentic responses and will bias the insights outcomes.

How to Avoid Biased Questions: Smart research design should never be ignored. Ask one question at a time and keep questions neutral to allow for honest responses.

Implications of Bias

The goal of market research is to gain true insights into a market and leverage those insights to make informed decisions that result in positive business outcomes. Bias can come from many places. Often companies rely heavily on their internal knowledge, gathering market research from the sales team or other customer-facing teams, which can be helpful but should usually be taken with a heaping teaspoon of salt.

While internal insights can be useful, they should be balanced by responsible market research — research that considers types of bias and employs proper methodologies to avoid them. Doing so will lead to insights that more accurately reflect reality and limit misinformation, guiding research sponsors toward the best possible outcome.