Survey Design Best Practices

What is Survey Design and Why Is It Important?

Survey design is the practice of creating a structured questionnaire that captures valuable insights and data from respondents. Tactical survey design methodology and using proper survey design best practices can mean the difference between a great survey experience and a truly terrible one.

So, you’ve decided to run a survey. You are confident it’s the best way to gain insight into your research project. You sit down to write the questionnaire, but you’re stuck. You know the questions you want to ask, but don’t know how to ask them. You’ve heard that survey design is important, but don’t know where to start.

Luckily, there are a few basic survey design principles that can elevate your survey from solidly mediocre to just plain solid.

Survey Design Best Practices to Remember When Running a Survey

1. Clearly Define Your Objective

Your objective is your guiding star. It informs how you structure your questions, the specific language you use, and ultimately what data you get at the end. With your objective in mind, you can focus on what exactly the survey needs to accomplish. Typically, the goal of a survey is to collect data that can be analyzed for actionable use. If your data doesn’t enable confident action, then the survey has failed its objective.

2. Always Include Survey Screening Questions

Whether you are surveying CEOs or consumers, a screening section, consisting of 5-10 questions, will ensure that you are getting the right people to answer the right questions. What are the defining factors that make an ideal respondent? What is their role, their seniority, and their experience with a given product? Think of the screener like a funnel, with the broadest questions around industry and geography at the top, and more detailed questions around job responsibilities or familiarity with the survey topic near the end. Be mindful of people’s time and terminate them immediately if they’re not eligible for the survey.

Another key function of the screener is that it hosts your quotas. For example, if companies with 5,000 full-time employees (FTE) are interesting for your client but you want only a small percentage of them, include a cap for that group’s respondents in your survey. As a rule, screening questions with Yes/No answer choices should be avoided. Instead, offer a range of potential answers and allow the respondent to qualify by selecting the correct response. For example, instead of asking “Are you a CEO?” you could ask “What is your role?” and have CEO listed among 4-5 other roles.

3. Implement Survey Logic Whenever Possible

Sound survey logic improves a respondent’s experience dramatically. Survey logic allows respondents to only see questions and/or answer choices that are relevant to them, based on how they answered previous questions. There are few things more frustrating in a survey than being forced to answer questions that aren’t relevant. If you need a respondent to remember a previous statement, remind them using survey logic. Human memory is fallible, and a well-placed reminder can reduce the most frustrating of survey outcomes: inconsistent responses.

Pathing can be a sensible option if you’re looking at multiple groups in your survey. For example, if you’re running due diligence on a healthcare product and are interested in finding out more information around users and nonusers, create separate pathing for them with two distinct question sets. The screener can help you identify who belongs to which group, and you can build out programming logic to send participants to the questions only relevant to their group.

4. Mutually Exclusive and Comprehensive Answer Options

It should never be ambiguous about how to answer a question. All answer choices should be clearly defined, comprehensive in nature, and mutually exclusive. There should always be a relevant answer choice. If there’s no way to be comprehensive, then it’s customary to include an “Other” option where the respondent can fill in their own response.

5. Consistent Scales

For surveys that contain multiple rating questions, keep the scales consistent. Each type of scale has its own merit, whether it’s a four-point scale, a seven-point scale, or a 10-point scale (or any other type of scale, for that matter). Whatever you determine to be best for the research objective, keep it consistent throughout the survey. It improves the respondent experience by making it easier to answer questions. Well-designed, easy-to-answer questions will produce good data.

6. Simple Language

Keep it simple! It works in many other areas of life and works in surveys, too. Remember that your population will define what “simple” language means. Speak the language of your specific audience and they will thank you by providing good data – and by not dropping out of your survey. Additionally, if respondents are confused about what is being asked, it could impact the data.

Don’t ask double-barreled questions, such as, “Which of the following brands are you aware of and which would you recommend?” Awareness and recommendation are two different topics, and they should not be combined in the same question. Just because a respondent is aware of a brand doesn’t necessarily mean they would recommend it.

7. Take Advantage of Varied Question Types

Single-select, multi-select, rating, ranking, grid, dropdowns – the list goes on. There are many ways to ask a question but knowing the right application of each is the differentiating factor. The right question type is easy to interpret and provides data that is easy to analyze. Remember to limit open-ended questions. Nothing fatigues a respondent faster than too many essay-style responses. You can get good data from two or three open-ended questions, but more than that and you will see a decline in quality and/or higher dropout rates. Another key consideration to consider is that surveys are often taken on mobile devices, so it is important to be mindful of the formatting of questions, limiting matrixes or grids which may be difficult to view on a mobile screen.

8. Be Thoughtful About the Number of Questions You Ask

Like screening, questions should be structured like a funnel with broader topics first, focusing on things such as overall budget and general key purchasing criteria. Those should be followed by more granular information, such as specific spending and key purchasing criteria by vendor. Finishing one topic before moving on to another can help to keep the respondent engaged and minimize any respondent confusion.

Your survey should ask the least number of questions possible that will still allow you to gain meaningful insight into your research objective. That number may be five questions, it may be 35 questions. Different populations have different tolerances for the length of a survey. Know what that tolerance is and adhere to it. And perhaps most importantly, don’t ask redundant or irrelevant questions.

Survey Design Metrics

Finally, in addition to these survey design best practices, it is important to consider two key metrics. The first is incidence rate, the rate at which respondents qualify for the screening section of the survey. The rate can have a direct effect on the feasibility for a survey. One way to increase the incidence rate is to expand and broaden your screening criteria. Allowing additional respondents to qualify can certainly be helpful to increase your incidence rate and broaden your questions.

The other metric is dropout rate, which looks at participants who have “dropped” out of, or exited, the survey without completing it. If the rate is high, it could be a result of something such as a programming issue, poor survey design, too many open-ended questions, or length. These respondents very likely won’t reengage with the survey.

If you follow these survey design best practices and princples, your surveys are more likely to deliver quality data that translates into actionable results.

---

Check out the other articles in our Survey Series:

• What Type of Survey Do You Need?
• Are You Running the Right Survey for the Wrong Reason?
• Why the Screener Section of Your Survey Is Compromising Your Results

---

About the Author

Will Mellor leads a team of accomplished project managers who serve financial service firms across North America. His team manages end-to-end survey delivery from first draft to final deliverable. Will is an expert on GLG’s internal membership and consumer populations, as well as survey design and research. Before coming to GLG, he was the VP of an economic consulting group, where he was responsible for designing economic impact models for clients in both the public sector and the private sector. Will has bachelor’s degrees in international business and finance and a master’s degree in applied economics.

---

If you have a question about GLG Surveys, contact us using the form below to speak with a member of our survey team.