data collection tips

How to Write Great Survey Questions

When starting a survey project, most people look forward to the fun, creative part of writing the questions. However, it doesn’t take long to realize that writing great survey questions is not as easy as it looks. Questionnaire design is more science than art – requiring critical attention be paid to question and answer order, structure and phrasing to ensure you get the reliable, quality feedback you are looking for.

A simple question, such as “How much did you enjoy the program?” could wreak havoc in your results, because it is inherently biased towards a positive response. “How did you feel about the program?” would be a more effective approach. Other pitfalls include asking multi-part questions, having overlapping answer choices, or asking the more difficult questions too early.

But have no fear. Outlined below are the basic principles of questionnaire design, along with some helpful tips, that will have you writing great, effective survey questions in no time:

Before You can Start Writing Great Survey Questions 

    • Know your objectives. Write down the purpose of your survey scanning, what information you need, and how you plan to use the data.
  • Work backwards. Make a list of the specific answers you need first, and then use that to drive your questionnaire.

Basic Guidelines for Writing Great Survey Questions

    • Keep questions focused. Make sure each question is designed for specific feedback. Avoid double-barrel questions like “How do you feel about our products and services?” as some respondents will focus on products and others on services. Instead, separate them into two questions.
    • Put easier questions first. This will increase participation and establish trust. By getting comfortable with the survey research by answering a few less complex questions first, your participants will be more likely to answer the more complex or sensitive questions later.
    • Organize by topic. Similar questions should be grouped together so the questionnaire flows naturally.
    • Keep it short and simple. Questions should be short, focused, and easy to answer. This will ensure a higher response rate and limit survey fatigue.
    • Be consistent. Use uniform rating scales, word choices and definitions throughout your survey. If you start with 1=low and 5=high, stick with that format.
    • Be precise. Avoid generic answer choices like “sometimes” and “rarely”. Use actual numbers instead (e.g, “more than 3 times per week”).
    • Be balanced. Provide an equal number of positive and negative response options.
    • Be complete. Include all possible answers, and make sure there is no overlap between answer options.
  • Eliminate bias. Try to construct the questions as objectively as possible. Avoid leading questions like, “Can you see why this product was voted best in customer satisfaction?” Instead, ask how they would describe their satisfaction level.

Common Question Types

Survey questions fall into two categories: Structured (fixed response) where they choose from a provided list of answer options and Non-structured (open-ended) where they can fill in their own text or numeric answer. Both are extremely useful, depending on the type of feedback you need.

Following are the most commonly used question types:

Multiple Choice

These are questions with two or more answer options. These are useful for collecting structured responses.

Single Response Style (select one answer)

Example 1: Do you smoke?     Y / N

Example 2:  If yes, how many cigarettes do you smoke per day?

use multiple choice to write great survey questions   

A common pitfall here is missing a possible response. Depending on your question, you may need to add a choice called “none”, or if you would like additional details, you could try an “other” option with space for a written response. You also want to make sure there is no overlap, such as using 10-20, 20-30, etc. in the previous example, which would clearly taint the results.  

Multiple Response Style (you may select more than one answer)

Example 1:  What is your race? (check all that apply)

use multiple response style to write great survey questions

Rating Scales

Rating scales ask respondents to rate how much they agree with a certain statement using a common scale (e.g, 1 to 5, where 1=low and 5=high). These are useful for gauging their opinions, attitudes and behaviors. When using rating scales, it is important to make sure you have a neutral option and a balanced, equal number of positive and negative responses. Scales most commonly use 5 or 7 options.

Example 1: The teacher was knowledgeable.

Example 2:  How would you describe your experience navigating the instruction manual.

use rating scale to write great survey questions (sample 2)

Common pitfalls here include being inconsistent with your scales (leading some respondents to answer incorrectly) and asking leading questions, such as, “We pride ourselves on our easy-to-use manuals. How easy was our manual to read?”

Ranking Scales

These ask respondents to rank a list of items in order (e.g, from favorite to least favorite, or most important to least important). It is recommended that you use these with caution. They are known to be reliable at determining first and last place, but not so much the fuzzy middle, as respondents often have to choose a pecking order for items that are essentially of equal value to them.

Example 1: Please rank the following customer service features in order of most to least important when contacting our agency by phone (1=most important, 5=least important)

use ranking scale to write great survey questions

Open-Ended Questions

These are questions with no provided answers options. Respondents answer by writing in their own text. These are great for eliciting responses about attitudes and opinions in a respondent’s own words, or having them provide a numeric answer without a suggested range. The downside is it requires extra time, can cause survey abandonment, and makes data collection and analysis more challenging.

Example: Name two ways we could have improved your customer experience today?

Questionnaire integrity is critical for getting quality data. By following these tips and guidelines, you will be well on your way to success. 

For more information on question and survey design or any aspect of survey mail management, contact us today!

By |2019-03-20T10:51:08+00:00December 6th, 2018|Survey Research Services|0 Comments

When to Do a Multimodal Survey?

Multimodal or mixed-mode surveys are research surveys that use two or more forms of communication to reach respondents (e.g, telephone and email). In today’s increasingly complex, interconnected world, we now have ways of communicating that didn’t exist even a few years ago. The list of channels seems almost endless nowadays, including regular and express mail, email, online, social media, mobile (text, instant message), scannable paper, telephone, kiosk, tablet, in-person, video, and more.   

So how do you know which channel or combination of channels is right for your survey project? The answer lies in the target audience you are trying to reach (as well as time and cost considerations). For example, older respondents are typically less trusting of online channels and can be reached more reliably by landline telephone and regular mail. Millennials may not even have a landline and would be more receptive to an email or a text message. Teens are generally receptive to mobile and social media. One constant to be mindful of, however, is that everyone has a physical address where they live – making mail a preferred channel in almost any multimodal research effort.

Not only do various target populations have preferred forms of communication, but there are also subsets within them that prefer something else entirely. Mobile phone call vs. text message among Generation X’ers is a prime example, in which someone’s preference is highly personal.

You should do a multimodal survey if you have a target population with varied respondents or hard-to-reach respondents.

Not only can a mixed methodology approach to data collection help you reach more respondents, but it can also help you maximize response rate. That’s because multiple channels give you more opportunities for follow ups, reminders, and options to complete the survey in a format that suits them.

As a leading provider of mail and multimodal surveys, we manage mixed methodology research every day, including planning, production, distribution, fulfillment and data collection. That said, our most popular service by far is Mail to Online. In this strategy, respondents are notified by regular mail and given instructions to complete the survey online. Respondents then have the option to print out or request a paper survey and complete a hard copy or complete the online survey.

Multimodal survey planning and execution requires a high level of expertise to run seamlessly. Your survey provider can help you manage all the details, including which channels to use, along with the projected cost and timeline.

For more information on multimodal surveys or any aspect of survey management, contact us today!

By |2019-03-20T10:54:13+00:00November 21st, 2018|Survey Mailing Services, Survey Research Services|0 Comments

This Mail Survey Spec Sheet Is Everything

Mail surveys are among the most effective tools in the research industry, besting online, email and phone survey methods in both response rate and data integrity. As with all survey research, however, survey success begins with careful planning, including brainstorming your goals, creating questions, choosing participants, determining your budget, and perhaps most tedious, choosing materials and pricing out costs with your vendors. But have no fear. We have you covered with these handy Survey Mail Spec Sheet and Timeline Templates that make your life easy and get you the honest, apples-to-apples quotes you need to keep your costs in check.

Vendors typically require a spec sheet in order to provide you with an accurate quote. The important thing here is to provide each vendor with the same spec sheet, so you can get an accurate read on their pricing.

Spec Sheet

Your spec sheet should document the list size, number of mailings, type and quantity of materials that will be needed, budget and requirements for each mailing. The good news is we’ve done the heavy lifting for you. This spreadsheet will help organize and calculate the materials across your mailings. Download the spec sheet.

Once your project specification sheet is completed, you’ll want to have your stakeholders, if applicable, review to make sure that everything has been captured. Now you can piece out the specification sheet and send to your different vendors for pricing and don’t forget to ask them for their timeline, which we’ll discuss next.

Timeline

Timeline is driven by the requirements of your project. For example, mailing 500 pieces vs. 50,000 pieces could have a significant impact to your schedule. Alternatively, the complexity of your mailing can also have a significant impact to your timeline. Mailing 10,000 units that have 5 pieces being inserted with a 4 way match, could take just as long to prep 50,000 pieces with 1 component and no matching. A best practice is to provide each vendor with a detailed timeline in advance so they know what your expectations are. We have filled in some ‘standard’ timelines for the different tasks but you can easily update with your requirements. Just populate your tasks and the number of working days required for each task and the spreadsheet will update accordingly. Download the timeline template.

With careful planning, you can account for all the variables in your mail survey, leaving little room for miscommunication and most importantly, get the accurate quotes you need to move forward with confidence.

If you’d like more information on mail survey planning or any aspect of survey management, contact us today!

By |2019-03-20T10:55:24+00:00November 8th, 2018|Survey Mailing Services|0 Comments