When starting a survey project, most people look forward to the fun, creative part of writing the questions. However, it doesn’t take long to realize that writing great survey questions is not as easy as it looks. Questionnaire design is more science than art – requiring critical attention be paid to question and answer order, structure and phrasing to ensure you get the reliable, quality feedback you are looking for.

A simple question, such as “How much did you enjoy the program?” could wreak havoc in your results, because it is inherently biased towards a positive response. “How did you feel about the program?” would be a more effective approach. Other pitfalls include asking multi-part questions, having overlapping answer choices, or asking the more difficult questions too early.

But have no fear. Outlined below are the basic principles of questionnaire design, along with some helpful tips, that will have you writing great, effective survey questions in no time:

Before You can Start Writing Great Survey Questions 

• • Know your objectives. Write down the purpose of your survey scanning, what information you need, and how you plan to use the data.

• • Work backwards. Make a list of the specific answers you need first, and then use that to drive your questionnaire.

Basic Guidelines for Writing Great Survey Questions

• • Keep questions focused. Make sure each question is designed for specific feedback. Avoid double-barrel questions like “How do you feel about our products and services?” as some respondents will focus on products and others on services. Instead, separate them into two questions.

• • Put easier questions first. This will increase participation and establish trust. By getting comfortable with the survey research by answering a few less complex questions first, your participants will be more likely to answer the more complex or sensitive questions later.

• • Organize by topic. Similar questions should be grouped together so the questionnaire flows naturally.

• • Keep it short and simple. Questions should be short, focused, and easy to answer. This will ensure a higher response rate and limit survey fatigue.

• • Be consistent. Use uniform rating scales, word choices and definitions throughout your survey. If you start with 1=low and 5=high, stick with that format.

• • Be precise. Avoid generic answer choices like “sometimes” and “rarely”. Use actual numbers instead (e.g, “more than 3 times per week”).

• • Be balanced. Provide an equal number of positive and negative response options.

• • Be complete. Include all possible answers, and make sure there is no overlap between answer options.

• • Eliminate bias. Try to construct the questions as objectively as possible. Avoid leading questions like, “Can you see why this product was voted best in customer satisfaction?” Instead, ask how they would describe their satisfaction level.

Common Question Types

Survey questions fall into two categories: Structured (fixed response) where they choose from a provided list of answer options and Non-structured (open-ended) where they can fill in their own text or numeric answer. Both are extremely useful, depending on the type of feedback you need.

Following are the most commonly used question types:

Multiple Choice

These are questions with two or more answer options. These are useful for collecting structured responses.

Single Response Style (select one answer)

Example 1: Do you smoke?     Y / N

Example 2:  If yes, how many cigarettes do you smoke per day?

A common pitfall here is missing a possible response. Depending on your question, you may need to add a choice called “none”, or if you would like additional details, you could try an “other” option with space for a written response. You also want to make sure there is no overlap, such as using 10-20, 20-30, etc. in the previous example, which would clearly taint the results.  

Multiple Response Style (you may select more than one answer)

Example 1:  What is your race? (check all that apply)

Rating Scales

Rating scales ask respondents to rate how much they agree with a certain statement using a common scale (e.g, 1 to 5, where 1=low and 5=high). These are useful for gauging their opinions, attitudes and behaviors. When using rating scales, it is important to make sure you have a neutral option and a balanced, equal number of positive and negative responses. Scales most commonly use 5 or 7 options.

Example 1: The teacher was knowledgeable.

Example 2:  How would you describe your experience navigating the instruction manual.

Common pitfalls here include being inconsistent with your scales (leading some respondents to answer incorrectly) and asking leading questions, such as, “We pride ourselves on our easy-to-use manuals. How easy was our manual to read?”

Ranking Scales

These ask respondents to rank a list of items in order (e.g, from favorite to least favorite, or most important to least important). It is recommended that you use these with caution. They are known to be reliable at determining first and last place, but not so much the fuzzy middle, as respondents often have to choose a pecking order for items that are essentially of equal value to them.

Example 1: Please rank the following customer service features in order of most to least important when contacting our agency by phone (1=most important, 5=least important)

Open-Ended Questions

These are questions with no provided answers options. Respondents answer by writing in their own text. These are great for eliciting responses about attitudes and opinions in a respondent’s own words or having them provide a numeric answer without a suggested range. The downside is it requires extra time, can cause survey abandonment, and makes data collection and analysis more challenging.

Example: Name two ways we could have improved your customer experience today?

Questionnaire integrity is critical for getting quality data. By following these tips and guidelines, you will be well on your way to success. 

For more information on questionnaire and survey design or any aspect of survey mail management, contact us today!