When the topic of a survey is sensitive or extremely private, the participants’ identity can be an important issue. In such a situation, anonymity and confidentiality may significantly affect the response rate. In a mail survey, some may wonder how an anonymous study, wherein not even the researcher knows the respondents’ identity, is possible when there is a need to send follow-ups, such as reminder postcards or secondary survey packets. Indeed, it is achievable using two methods. First, researchers can decide to send mail follow-ups to the entire sample. Second, for a more cost-effective technique, researchers may guarantee confidentiality to respondents by promising not to reveal the information to anyone. In this post learn the Effect of Numbering Mail Survey Return Envelopes.

Follow-ups can be implemented using tracking numbers on the questionnaires or envelopes instead of using respondents’ names. Consequently, it is vital to explain to the respondent why the tracking number is present and how it will be used. However, in some situations, even the numbering of mail survey return envelopes can become an issue. Some people may feel that the tracking number can link the answers back to them, thereby choosing not to participate, skipping the delicate questions, or modifying their answers, consciously or unconsciously, to make them more inconspicuous or tolerable. 

In a 2014 study, “Effect of numbering of return envelopes on participation, explicit refusals, and bias: experiment and meta-analysis”, the researchers examined the impact of numbering return envelopes on the participation and the results of a survey on a sensitive topic among hospital staff. 

In this study, two consecutive samples were drawn from the same randomly ordered sampling frame. The two samples were contacted at different moments in time, two months apart. The study population included all doctors and nurses. The two randomly drawn subsamples were provided numbered (N = 1100) and non-numbered (N = 500) envelopes to return completed questionnaires. The researchers compared participation, explicit refusals, item responses, and conducted a meta-analysis of the effect of questionnaire/envelope numbering on participation in health surveys.

Significant Results of the Study:

1. The participation rate was lower in the “numbered” group (30.3%) compared to the “non-numbered” group (35%). 

2. The proportion of explicit refusals was higher in the “numbered” group (23.1%) compared to the “non-numbered” group (17.5%).

3. The meta-analysis of 7 experimental surveys (including this one) indicated that numbering is associated with a 2.4% decrease in the survey response rate.

The study shows that in some instances, numbered return envelopes can reduce the response rate, increase explicit refusals to participate in a sensitive survey, and lower participation. 

All in all, researchers should aim to keep the privacy of participants. However, when deciding whether to be completely anonymous or use the numbering method, researchers should weigh if a 2 to 3 percentage point difference in response rate is significant enough to affect their research results. A small reduction in the response rate may be an acceptable price to pay for the benefits, such as reduced cost and avoiding annoyance from respondents who will receive follow-ups after already completing the survey. However, in situations where increasing the response rate and the target population’s accuracy is vital, not using a respondent ID may be needed.  

For more information on mail survey, survey management, and data collection in general, contact DataForce!