Primary research is a valuable tool in better serving your audiences through your digital platforms. However, research isn’t always representative of the groups that you are hoping to serve for a number of reasons — whether that is a bias of the researcher, collection methods, or hesitation on the part of the audiences themselves. Collecting diverse audience perspectives can help you gain a more holistic and inclusive understanding of your stakeholders and, ultimately, extend the opportunity for more people to take part in your organization’s mission.

What causes bias in research?

While research is meant to include a diverse set of perspectives or samples in order to be fully representative, bias can get in the way. There are two types of bias that typically present themselves in research: researcher bias and respondent bias. 

Researcher bias occurs when a research tool is written in such a way that it sways a respondent’s answers. The most common types of bias in writing a research tool include: 

• Confirmation bias: When a researcher forms a hypothesis or belief and heavily weighs responses that confirm their hypotheses and dismisses evidence that doesn’t. 
• Culture bias: When a researcher makes assumptions about motivations and influence through a cultural lens. More specifically, when research is used to judge a culture other than the researcher’s by the values and standards of their own.
• Question-order bias: This occurs when one question influences answers to following questions, and is generally a very difficult one to avoid.
• Leading questions: When a researcher develops leading questions because they are trying to confirm a hypothesis, or overestimates their understanding of the audience by using their own words or connecting feelings to behaviors without confirmation.

Respondent bias occurs when a respondent has formed an opinion about the researcher, organization, or language upon which they are being asked to provide feedback. A few reasons that your research may fail to reach or collect feedback from a diverse set of perspectives may include cultural differences, linguistic differences, financial and time constraints, logistical challenges, lack of incentive, reluctance and distrust based on past unethical practices, and potential fear of exploitation. 

Another type of bias that applies to both researchers and respondents is the halo effect. This is the tendency to see something or someone in a certain light because of a single, positive attribute. This affects researchers in that they may view responses based on one single positive response. The halo effect can also take place when a respondent tries to answer in the affirmative because they have a positive view of the organization or moderator. This is particularly true in the case of audiences with different levels of acculturation.

How to reduce bias and increase diversity

There are a number of ways by which you can reduce bias and hear from a more diverse research sample. Even if you have limited time or resources at your disposal, here are four things to consider when attempting to collect a more representative set of perspectives.

1. Clearly define your target audience

Understanding who you are trying to reach is a crucial part of your mission in collecting diverse perspectives. Ideally, you want to include a broad cross-section of your audience but that is not always possible due to time and budget constraints. You can start by considering what type of audience information might already exist within your organization to help you reach a broad audience and ensure demographic and geographic diversity — such as email addresses, physical addresses, demographic background information, or engagement data. 

If you are hoping to extend your reach or hear from audiences that are not currently captured by this data, consider how you can use external ethnographic researchers to help gather information from additional audiences. Engaging with external consultants can help you broaden your reach and formulate a more inclusive research process. For example, if your data collection sources tell you that your current communications are primarily engaging white, suburban moms, an external researcher may help you broaden that audience to BIPOC or Gen Z by building out tools and recruitment strategies that will capture a wider audience.

Also consider what screening criteria should be included to ensure proper representation, without being too broad or specific about things like age, location, gender, race, or other demographics. Creating a screener will help you ensure that you are capturing feedback from the range of respondents you are targeting and giving you the data you need to confidently slice and dice your data to find trends amongst particular audiences.

2. Tailor your collection methods

Using the same tools for every audience will ensure that you capture a fairly homogeneous set of data. Because factors such as economic disparity, access to technology, and demographic trends dictate which tools audiences might have access to or prefer to use, there are notable differences in areas like internet access and preferred communication methods based on geographic location and ethnicity. This means that you may not hear from a diverse sample if you do not consider how audiences are most likely to give feedback. 

Based on the audiences you want to capture, consider if your survey, interview methods, or other data collection tools are optimized for the technology those audiences have access to or prefer. For example, some audiences may prefer to answer a survey via a mobile device while others may be more open to sitting at their laptop. Some (typically younger) audiences may be more liable to respond to the initial recruitment if you target them via social media channels instead of over email while others prefer to click through a link sent in a weekly update. 

3. Consider language & cultural differences

There is evidence that shows that responses to primary research can be influenced by a person’s level of acculturation. This might mean that respondents may have a bias toward answering in a particular way based on the language and words used in data collection. Depending on whom you are collecting responses from, you may consider providing participants the option to complete the survey in their native language, particularly in the case of phone studies and in-depth interviews. Ensure that translations are authentic and avoid overly formal language so that they are accessible to a wide audience.

If you are creating a research tool that is entirely for an English-speaking audience, also consider how you can formulate questions that are simple and easy to understand. Part of collecting diverse perspectives means creating research tools that are usable by a variety of ages, occupations, educational levels, cognitive abilities, and geographic locations. Using words or phrases that are too jargony, regional, or complicated makes it more difficult to capture perspectives of those that the language excludes.

4. Provide an incentive as a token of appreciation

People can’t always justify taking the time out of their schedules to provide their opinions for free. An incentive for people’s time can be a good way to motivate people beyond just those who are the most engaged with an organization or cause, however, you should afford appropriate forethought when considering how much or when to offer an incentive.

Whether or not to provide financial incentives to research participants remains a highly-contested issue. Those who argue against incentives see the practice as coercive, which skews motivations for decision-making in responses and therefore compromises the integrity of findings. Those in favor of incentives argue that they can recognize and humanize the participant contributions and that payment is a necessary means of increasing recruitment.

If you decide to offer a financial incentive for your participants’ time, make sure that it is appropriate within the parameters of what you are asking them to provide. Consider your incentive as more of an appreciation than a payment. Consider a virtual reward (such as a gift card code) that they can redeem instantly such as a $15 online retail gift card. The incentive should also have broad appeal to a variety of audiences and is cost-effective for your organization. 

Resources

Looking for more in-depth tools on bias in researching and collecting diverse perspectives? Here are a few resources that may prove helpful: