Questionnaire Design
If your research project is going to involve the use of a questionnaire, then it’s essential to put time and effort into getting the format right. The first step is to determine precisely what information you need to know, while thinking carefully about your hypothesis. Although it can be important to include relevant background questions, you need to make sure that the questions reflect the aims of your project and that you don’t collect unnecessary data. Attempt to write out your questions. Do they relate to your hypothesis? If they don’t, get rid of them.

You should be aware of the problems caused by questions that create an attitude, known as ‘rarification’, and you must also remember that what people tell you in answer to a question does not always reflect their actual behaviour. These are two of the most common mistakes in questionnaire design. The Pharmacy Practice Research Resource Centre produced a very useful bulletin on designing and administering questionnaires. We are grateful to the Department of Health for allowing us to reproduce some of the content here.

Question type
Decide on the response format. Is it going to be opened or closed? A closed question provides a number of alternative answers from which a choice has to be made. An open-ended question allows the respondent to formulate their own answer. There’s no right or wrong approach. Your decision will be based on respondent motivation, method of administering the questionnaire, the topic covered, expertise and time spent developing a good set of unbiased responses. Each has advantages and disadvantages.

Closed questions
There are a variety of approaches in the use of closed questions. The following are examples of the ones most frequently employed:

1. Two-way Question: Here there are only two alternatives: yes/no, good/bad, for/against and so on.

2. Likert Scale: Here you provide your respondent with statements and ask them to agree or disagree. The scale usually comprises 5 points but there may be more or less. Whether to offer a middle ‘no opinion’ type category is an issue of some controversy. Scales generate ordinal data and this has implications for statistical analysis.

3. Semantic Differential Scale: Here you provide the respondent with a scale featuring a pair of diametrically opposed adjectives at either end. The respondent puts a mark somewhere between the two extremes. The number of points on the scale can vary but no more than ten points are usually recommended. Be aware of the phenomenon known as ‘the error of central tendency’ - respondents are often afraid of using the extreme categories.

4. Checklists: Here the respondent has a set of items and is asked to circle/tick each relevant one. All responses will have to be tabulated, so consideration of the analysis stage is again very important with this choice of format.

5. Ranking: A set of items is provided and the respondent is asked to list them in order of preference, importance, merit, etc. No more than ten items to be ranked should be used. Remember that ranking does not tell you anything about the distance between the ranks.

Wording the Questions
You will need to make sure that questions are clear, unambiguous, and useful. The wording is fundamental to both the validity and reliability of any study. Always pilot and evaluate your questions first. Keep the following points in mind:

• Don’t use jargon or abbreviations.
• Keep questions simple and as short as possible.
• Don’t use vague terms. Avoid ambiguity. Be precise.
• Avoid ‘loaded’ or ‘leading’ questions.
• Avoid ‘double-barrelled’ questions.
• Avoid ‘double-negative’ questions.
• Use common concepts.
• Take care over questions that involve memory/recall.
• Take care over the use of hypothetical questions.
• Take care when covering embarrassing or sensitive issues.
• Avoid using negative words or implicit negatives.
• Avoid ‘presumption’ questions.
• Watch out for prestige bias in the question.

Appearance and Layout
Good design can be helpful in increasing your response rate. Appearance, layout and length will depend on how you are going to administer the questionnaire. How much money you have available for your research will obviously have a bearing on this, but there are a variety of ways to improve the appearance of a questionnaire. Here are a few points to bear in mind:

• Make the questionnaire attractive
• Use space generously; avoid a cramped, untidy appearance
• Make headings and instructions clear
• Use coloured paper
• Make sure the method of answering is obvious
• Include code boxes if necessary, making sure they don’t interfere with readability
• Don’t split a question between two pages
• Number all questions
• Take care over question order. Generally start with broad, straightforward ones and include more complicated, specific or sensitive ones later
• The questions should proceed in a logical manner
• Vary the question format to add interest
• End questionnaire with a “Thank you”

Piloting the Questionnaire
It is absolutely crucial to pilot your questionnaire. You will want to test how long it takes to complete the questionnaire, check that all questions and instructions are clear and try to expose any items that will not generate usable data. Piloting will also develop your interviewing skills, so ideally respondents in a pilot should be as similar as possible to those in the main study. In practice it is not always possible to pilot. If a pilot is not feasible, try the questionnaire out on friends or colleagues. You could ask them:

• How long did it take to complete?
• Were the instructions clear?
• Were any questions ambiguous?
• Were any questions objectionable?
• Was the layout clear and easy to follow?
• Were any topics omitted?

Bias
Bias can creep in for a number of reasons and can depend on the way you administer the questionnaire. If you use an interviewer, rather than have the questionnaire self-completed, there is a high potential for interviewer bias. Interviewers can influence the responses in many ways, even by their tone of voice. A ‘response effect‘ can arise, for example, out of the eagerness of the respondent to please the interviewer or from a tendency by the interviewer to seek out answers that support preconceived notions. It is far easier to ‘lead’ in an interview than it is in a questionnaire. The advantage of using an interviewer is flexibility. Interviewers can probe deeper, build rapport, put respondents at ease and keep them interested. An interviewer will need to be trained, however.

If the interview is unstructured or semi-structured, reporting respondent’s answers verbatim is vitally important. Timing and venue for an interview can also lead to bias. With a self-completed questionnaire, interviewer bias is obviously eliminated, but other types of bias can creep in. For example, respondents can answer questions not in the intended order or other people, for whom the questionnaire was not intended, can answer the questions instead.

Response bias is more pronounced with self-completion, postal questionnaires. Non-response is even higher and is not a random process. Knowing who your non-respondents are is vital if any decision about possible bias is made. Ideally your response rate should not be lower than 66%. Reminders and second questionnaires will increase the response rate.

Administering the Questionnaire

By post? By you? By another interviewer?

If you decide on a postal survey, include an SAE; it will improve the response rate. Include a covering letter explaining the purpose of the study. Give a guarantee of confidentiality and/or anonymity and tell the respondent how s/he was selected. If you have official ethical approval, say so. Usually it is advisable to give as much information in the letter as possible. Say when you would like the questionnaire returned. Decide what you will do about non-respondents before sending questionnaires. You will not be able to send reminders if responses are anonymous, or if that promise was made.

Further Information
A very useful guide to questionnaire design from Leeds University can be found in pdf format at http://www.leeds.ac.uk/iss/documentation/top/top2. This has not been written specifically in the context of health research but it is clearly written and covers the most important do’s and don’ts of good questionnaire design. There is an excellent collection of more detailed web materials on the use of questionnaires in research collected by the University of British Columbia.