Choosing the right sample size is vital to accurate survey outcomes. With the right number of survey respondents, your target population will be represented with acceptable accuracy. Although it may seem like a simple undertaking, you must put a lot of thought into determining your survey’s most effective sample size.
In most cases, the larger your sample size, the more confident you can be that the responses you receive reflect the total target population. However, there is another side to that coin. Deciding on a larger sample size requires you to distribute more surveys, and that drives up the cost of your survey project.
It’s not as simple as randomly selecting a large sample size and assuming that will equate to successful response rates. The pitfall of using no forethought when selecting a sample size is that you may select a sample that is not statistically significant or will cause you to exceed your budget. To arrive at your optimum sample size, you must start by asking a few key questions.
1. How Large Is Your Target Population?
You can begin to understand your sample size needs by first determining the size of the total population of possible respondents. If you want to understand something more large-scale, such as how grocery shoppers are using mobile devices, you will need to use a larger sample size. That’s because there are millions of grocery shoppers who use mobile devices. On the other hand, if you want to assess a smaller, more specific group, such as the opinions of the faculty at a particular university, the population is limited to a few thousand or even a few hundred.
2. How Accurate Do You Need to Be in Choosing the Right Sample Size?
Because you are only using a small sample group out of a larger target population, you must now make some decisions about accuracy. Your accuracy will depend on choosing an acceptable margin of error and accepted confidence level.
Your margin of error measures how well the responses from your sample reflect the opinions of your entire target population. A commonly chosen margin of error is 5%. You must also keep in mind that lower margins of error require a larger sample size.
Your confidence level represents the degree to which your sample is an accurate sample of the population. If you decide your project requires a 95% confidence level, you are declaring that if you repeated your survey, you would get the same results 95% of the time. Higher confidence levels require a larger sample size.
3. What Is Your Expected Response Rate?
Arriving at an answer to the first two questions is relatively straightforward. First, determine the size of your target population. Then decide how accurate you want your survey results to be.
Now, on to the challenging part: estimating what your response rate will be. Estimating your response rate is often difficult; there are many factors that contribute to a respondent’s decision to complete and return your survey. The list of factors includes:
- Questionnaire length
- Question complexity
- Importance of the survey to the respondent
- Level of respondent interest in the survey
- Availability of incentives
Fortunately, you are in control of most of these factors. Depending on your target population and the decisions you make about how you will administer your survey, your response rate can range from 1% to 60%. Lower expected response rates require a larger sample size.
For more tips on Survey Design, check our blog post on How to Write Great Survey Questions.
Working with an Experienced Survey Administration Partner in Choosing the Right Sample Size
Your sample size has a direct impact on the validity and cost of your survey. Consequently, it’s crucial that you get that number right. Once you have determined the right sample size for your survey, there’s still more work to do. How will you distribute the surveys? How will you gather them and analyze the outcomes?
This is where your survey administration partner can add value to the process. When you work with an experienced vendor, they can consult with you about the nature of your survey research and suggest choices that will help you achieve your goals in an efficient and cost-effective manner.
Your partner can help you measure your target population and decide on an appropriate margin of error and confidence level. In addition—although no two survey projects are ever exactly alike—your partner can rely on their experience with working with a wide range of organizations and projects to help predict what your response rate might be.
When you are planning your next survey project—and trying to understand your optimal sample size—consider getting help from an experienced survey administration partner.