Gomez Research
Marketing Research for the Public and Nonprofit SectorsTips for Effective Questionnaire Design
A well-designed questionnaire is one of the most important, and often underestimated, steps in the survey research process. Each question should have a clear purpose, directly tied to the goals of the project, and must be designed in such a way as to elicit clear and truthful answers. Confused or biased questions will affect the reliability of survey findings. In designing a questionnaire, keep the following guidelines in mind.
1. Decide what the purpose of the study is before you design the questionnaire. Resist the temptation to include questions simply because they are "interesting." Instead, limit the questionnaire to only those questions that specifically relate to the study at hand. Without this discipline, questionnaires become long and unfocused, resulting in low response rates.
2. Be careful not to lead the respondent into providing a particular answer. Pay attention to any bias you might be introducing in the way you word a question. For example, by asking, "Do you agree that the mayor is doing good job?" you are communicating a bias in favor of the mayor's performance. If the respondent actually believes the mayor is doing a poor job, he or she might be reluctant to disagree with you.
3. Make sure you are not asking two questions at the same time. In an effort to shorten questionnaires, organizations often combine two questions into one. The difficulty is you don't know which question the respondent is answering. For example, an organization asks its customers, "Are you satisfied with the range of program services and hours of operation?" If the respondent answers "not at all satisfied," the organization will not know if the respondent is dissatisfied with the types of services being offered or the hours those services are provided.
4. Allow for all possible answers. When including a multiple choice question, it is important to include all possible answers so respondents aren't forced to give a false answer or to skip the question altogether. For example, an organization asks: "When was the last time you rotated the tires on your car? (a) Within the last year; (b) With the last two years; (c) More than two years ago." What if the respondent has never rotated his or her tires? What if the respondent doesn't own a car? These problems could be resolved by including additional screening questions, or by changing the structure of the original question to include more possible answers.
5. Ask questions that generate variability. When a question has too few categories, respondents often give the same, middle-of-the road answer. This does not provide new information or allow for analysis to examine differences among people or groups. For example, an organization asks, "How harmful do you consider alcohol to be? (a) Very harmful; (b) Somewhat harmful; (c) Not at all harmful?" Almost all respondents will give answer, "(b) Somewhat harmful." Instead, ask the question in a way that reveals the true variability in how people feel about a given issue.
6. Use simple language. Avoid unfamiliar words, vague language, and compound sentences. Simple language will keep questions clear and direct. More complex constructions won't improve the communication and are often used when researchers haven't narrow down what they are really trying to ask.
7. Keep the questionnaire short. Respondents develop fatigue and, after a certain point, won't provide you with thoughtful answers. Ideally, telephone surveys should be no more than 10 minutes long and an online survey should take 5 minutes. Longer surveys produce low response rates, which is one of the most important determinants of reliability. Response rates are critical in determining whether a sample is truly representative of the targeted population. When only a small proportion of a population responds to a survey it is highly probable that the individuals who did not respond differ in important ways from those who did respond-and may have provided very different answers.
8. Provide anonymity to respondents, if possible. To evoke truthful responses, it is important that respondents are not concerned about the consequences of their answers. Anonymous questionnaires that have no identifying information are more likely to produce honest responses.
