Questionnaire Design

Step1. Develop question topics (A rough set of topics/question areas to be addressed that can then be fitted into specific question formats.)

Researchers should take account of
( the project’s research objectives
The key driver of questionnaire design, determines the topics to be covered, which information is truly needed in comparison to information that is simply “nice to know”. 那些nice to know的问题，会增加问卷的长度，而这种不必要的长度会导致non response rates的增加。 ( the findings of any exploratory qualitative research undertaken 如果在quantitative之前进行过这种exploratory qualitative research，那么这种findings是有帮助的，可以帮助阐述应该问什么问题以及每一个topic的最好处理方式，同样也能帮助determine the relevant wording for questions and statements to be used in the rating scales. ( the characteristics of the respondents.

Respondents must be able to provide the information requested. Ability is based on the respondents having knowledge of the subject in questions (有些问题问的太精确，比如你上次去鞋店试了几双鞋这种，很难recall). –> so researcher needs to put himself/herself in the position of the respondent and then determine… Some questions are quite sensitive and private or boring and uninteresting –> researchers should determine how much and how specific these questions are needed. Make sure that the questions are relevent and interesting to potential respondents.

Step2. Select question and response formats

There are 3 main types of questions
Open-ended questions: Open-ended questions are unstructured questions that respondents answer in their own words. No pre-set choices of answers; respondents can decide whether to provide a brief one-word answer or sth very detailed. Advantages:

*The range of potential answers is quite wide, some answers may include the information or knowledge that researchers have not recognized before. It is quite hard to list all the proper answers. So this is a proper way. *It may help to explain the answers to other types of questions appearing in a questionnaire. Some attitude rating scales may show a person’s dissatisfaction of sth,but the reasons may be found or come from the open ended questions Drawbacks:

*It is quite hard to analyze and interpret. Cause the process involves editing and coding. Researchers should firstly editing the answers into a number of categories and then code each of the answers into one of these categories. —-> It can be overcome by pre-coding (before the interviews) rather than post-coding (after the interviews) potential responses. Researchers can put a list of potential answers with an “other” category, it is still an open-ended questions. This pre-coded open-ended question is different from the closed or multiple-choice questions because the list of answers is never read out or shown in pre-coded open-ended. *The depth of information may vary dramatically between different respondents. 比如shy or inarticulate respondents may provide the bare minimum of information.而且这种问题很可能在没有Interviewer的情况下，就空着了，那就是incomplete responses. Closed questions: require the respondent to make a selection from a predefined list of responses, specify the set of response alternatives and the response format. There are 2 main types of closed questions.

( Dichotomous questions:
*A dichotomous question has only two response alternatives: yes or no, agree or disagree, and so on. I.e. Have you shopped at Tesco before?
Yes 1
No 2
*Often, the two alternatives of interest are supplemented by a neutral alternative, such as “no opinion,” “don’t know,” “both,” or “none.” I.e. Do you intend to buy a new car within the next six months?

_____ Yes
_____ No
_____ Don’t know
(Multiple-choice questions:
The researcher provides a choice of answers and respondents are asked to select one or more of the alternatives given. I.e. Do you intend to buy a new car within the next six months?
____Definitely will not buy
____Probably will not buy
____Undecided
____Probably will buy
____Definitely will buy
____Other (please specify)
*Mutually exclusive
Each of responses should be distinct, with no overlap between categories. 例子：

*Collectively exhaustive
All potential responses should be listed. Researcher needs to know all or most of possible answers. This is relatively straightforward for factual information (such as newspapers read, brands of dog food purchased, European countries visited) where there is a relatively finite list. Even though researchers think they know all the potential answers, it is often best to include an “other” category. 缺陷：会使问卷看起来太长，导致response rates的下降；在telephone interviewing里不好操作。 Researchers要注意的：如果用software来制作，要注意设定的回答是single还是Multiple模式；还要注意每个小选项的排序也很重要，尤其是当respondents如确定回答什么的时候。调查表明最前面和最后面是最容易被选的，所以可以使系统自动给每个问卷排不一样的顺序，避免这种问题的出现。 Scaling questions

Questions that assign numerical measures to subjective concepts. (e.g., attitudes, opinions, and feelings) The information from different groups of the population is more easily compared and summarized by using this kind of questions.

A number of different dimensions in scaling questions:

(Unidimensional vs. Multidimensional assessment
*Unidimensional scaling focus on only one attribute, for example, satisfaction. Respondents would be asked to rate their satisfaction with a particular product or service. *Multidimensional scaling looks at a variety of dimensions, 比如了解对一个人对在hotel住的一晚是否满意，会从多个方面去调查，比如卫生，隔音效果等等。

(Graphic vs. Itemised rating formats
*Graphic rating scales is also known as continuous rating scales, respondents rate the objects by placing a mark at the appropriate position on a line that runs from one extreme of the criterion variable to the other. [pic]

*Itemised rating scales, the respondents are provided with a scale that has a number or brief description associated with each category. The categories are ordered in terms of scale position, and the respondents are required to select the specified category that best describes the object being rated. 包括三种：Likert, semantic, stapel.

-Likert scale: requires the respondents to indicate a degree of agreement or disagreement with each of a series of statements about the stimulus objects.

-Semantic differential scale: a seven-point rating scale with end points associated with bipolar labels that have semantic meaning. 中间是有个中立的选项”0”.

-Stapel scale: a unipolar rating scale with ten categories numbered from -5 to +5, without a neutral point (zero). This scale is usually presented vertically.
(Comparative vs. Non-comparative assessments
Comparative rating scale: It asks respondents to compare issue in relation to a common frame of reference. Non-comparative rating scale: It does not provide a standard of reference.

④Forced vs. Non-forced scales
Forced scale: does not allow respondents the option of selecting a neutral rather than a positive view about an attribute. non-forced scale: In situations where the respondents are expected to have no opinion, the accuracy of the data may be improved by a non-forced scale.

⑤Balanced vs. Unbalanced scales
Balanced scale: is one that has an equal number of positive and negative response choices. Unbalanced scale: can be useful when you know there will be an overwhelming response in a specific direction.

Define the issue in terms of who, what, when, where, why, and way (the six Ws). Who, what, when, and where are particularly important.

Which brand of shampoo do you use?(Incorrect)

Which brand or brands of shampoo have you personally used at home during the last month? n case of more than one brand, please list all the brands that apply.(Correct)

Step3. Select wording
The wording and phrasing should be as simple and straightforward as possible.

“Do you think the distribution of soft drinks is adequate?” (Incorrect)

“Do you think soft drinks are readily available when you want to buy them?” (Correct)

Use Unambiguous Words
In a typical month, how often do you shop in department stores?

_____ Never
_____ Occasionally
_____ Sometimes
_____ Often
_____ Regularly (Incorrect)
In a typical month, how often do you shop in department stores?

_____ Less than once
_____ 1 or 2 times
_____ 3 or 4 times
_____ More than 4 times(Correct)

Avoid Leading or Biasing Questions
A leading question is one that clues the respondent to what the answer should be, as in the following:
Do you think that patriotic Scottish people should buy imported cars when that would put Scottish people out of work?

_____ Yes
_____ No
_____ Don’t know(Incorrect)

Do you think that Scottish people should buy imported cars?
_____ Yes
_____ No
_____ Don’t know (Correct)

Avoid Implicit Alternatives
Implicit(不言明的/含蓄的) alternatives: An alternative that is not explicitly expressed in the options is an implicit alternative.

Do you like to fly when traveling short distances? (Incorrect) Do you like to fly when traveling short distances, or would you rather drive? (Correct)

Avoid Implicit assumptions
Questions with implicit assumptions are questions where the researcher and the respondent are using different frames of reference as a result of assumptions that both parties make about the questions being asked. 就是使用了假设，有可能对方并没有遇到过这种情况，但假设被访者有这种经历。

Avoid double-barrelled questions
questions where two topics are raised within one question.
Have u seen an improvement in the quality of this hotel’s food and accomodations? Yes_______ No________ Don’t know_________ (Incorrect)

Avoid Generalizations and Estimates
“What is the annual per capita人均 expenditure on groceries in your household?” (Incorrect)
“What is the monthly (or weekly) expenditure on groceries in your household?” (Correct)

“How many members are there in your household?” (Correct)

Step4. Determine sequence

Funnel sequence of questioning

In sequence questions in a questionnaire, it is often best to approach it from the respondent’s point of view: logical and interesting? Questions on similar topics should be clustered together within the questionnaire, allowing respondents to maintain their train of thought on one topic before moving on to the next. Classification questions是一种调查personal or demographic characteristics的问题，关于这种问题的位置问题是有争议的。可以放在问卷的开头，如果说这类问题用了选择人群，比如是否accept或者reject.也可以放在最后，考虑到个人的兴趣，因为一开始做问卷的时候先把难题复杂的给答了，到后面兴趣减退，就可以很轻松的回答这些难的题目。 这种funnel sequence of questioning是按照一种从general到specific的顺序。It is particularly critical where answers to earlier specific questions could bias the answers to later questions. It is also important where the researcher wishes to ensure that respondents are only asked questions that are specifically relevant to them.会有skip patterns.

Step5. Design layout and appearance

The key elements are:
Spacing; Quality of production; Variety; Coding/analysis requirements.

Specific design issues for online surveys:
Headings and progress information; Forced completion; The first page; The last page; Multimedia.

Step6. Pilot test
It is also known as pre-testing, it involves administering a questionnaire to a limited number of potential respondents in order to identify and correct design flaws.

Tips:
*A questionnaire should not be used in the field survey without adequate pretesting. *All aspects of the questionnaire should be tested, including question content, wording, sequence, form and layout, question difficulty, and instructions. *The respondents for the pretest and for the actual survey should be drawn from the same population. *Pretests are best done by personal interviews, because interviewers can observe respondents’ reactions and attitudes.