One of the most difficult parts of creating a survey is determining the questions and their exact wording. Even slight variations in the wording can lead to misinterpretation of meaning or confusion. This in turn can lead to biased or incorrect data collection, which we don’t want.
Welcome to part 2 of this article on Asking the Right Questions, where I’ll lay out more questions to ask yourself about your questions in order to make sure they are clear and respondent-ready. So, let’s get started:
When was this exactly?
Does your question depend on a certain timeframe? There’s a big difference between:
- Do you watch TV?
- Do you watch TV every day?
- How many hours a week do you watch TV?
You need to determine what kind of data you hope to glean and change the wording of your question accordingly. Note that the more specific you ask your respondent to get the more likely you are to collect “incorrect” data; this is also true of large time-frames. If you ask your respondent how many cumulative minutes per month they watch TV, you’ve just shot yourself in the foot. They’re never going to calculate that! It’s too specific and too large a time frame.
There’s also a difference between “in the past week” and “during the course of an average week”. Realize that if you ask a question with this first time frame you run the risk of getting period specific data. For example perhaps a respondent went camping the week before answering your questionnaire and therefore watched no TV whatsoever, but on average they watch 15 hours of TV a week. Or perhaps the week before you sent your survey the Olympics were on and everyone had their TVs on more than they usually would. That being said it may be easier for the respondent to calculate last week, rather than conceptualize how much they watch in the average week. You need to be conscious of the difference between these wordings in order to choose the one appropriate for the kind of data you are hoping to get.
What units do you want your respondents to answer in?
When you asked for how large their kitchen is did you mention whether you wanted the answer in square feet or square meters? If you asked about salaries, for example, did you want them to write 60 K or 60 000? Be sure that you note the units you want your respondent to use and, if appropriate, an example of what it should look like.
Are you making your respondent uncomfortable?
Are your questions too personal? Sometimes we ask questions that make respondents feel uncomfortable even if they are aware the questionnaire is anonymous (medical history, religion, sexual history …). A simple change in wording can turn a question from relatively impersonal to probing and inappropriate.
Keep social norms and morays in mind. Don’t ask questions that the respondent will feel are too personal to answer.
Are you being offensive?
Depending on who your respondents are you might need to double check whether your questions are “PC”. Some groups are easier to offend than others. Each culture, country, group, etc… has their own triggers that upset them. Be aware of this and make sure you aren’t upsetting anyone. You may get unwanted backlash from that kind of mistake.
Are you angling for an answer?
Are you asking loaded questions? Is your wording likely to get you biased results? For example, there is a subtle difference between “Please indicate how satisfied you are by our services:” and “Are you very satisfied by our services?” The second is slightly biased and will lead to different results from the more neutral first example.
By now you’re telling yourself: “My goodness! Since when has it been so complicated to ask a simple question?!” It’s a lot to take in, but the main thing to remember is that you need to read your question through the eyes of your respondent.
This list of questions you should ask yourself is simply a guide to help you optimize your questionnaires by starting with clear questions.
Even after asking yourself all these questions about your questions there will still be some trial and error. However, after a couple of questionnaires you’ll start to get the hang of it. You’ll become a super question writer and your colleagues will all come to you for help! (Make sure you charge them 5 cents for each question, you’ll be rich!) ;)
If you want information concerning this process you might want to take a look at William M.K. Trochim’s Research Methods Knowledge Base.
Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: <http://www.socialresearchmethods.net/kb/> (version current as of October 20, 2006).
Sign in and try it out for yourself!
Join us on