Using rating scale questions
There is not evidence to suggest that one rating scale model is better than another. Whether you prefer a 5-point scale to a 7-point scale should make no difference. It is, however, recommended that the scale starts with the least attractive answer and ends with the most attractive answer – in reading order, either moving across or downwards.
Sometimes it is not at all necessary to rate something, and you don’t need to use rating scales in these cases. In these cases a simple ”yes/no”, ”good/bad” is sufficient.
When this is said, however, Tivian recommends you use a number scale from 1-6 where 1 is the lowest and 6 is the highest value.
We do this for a number of reasons:
- ”Dice roll ratings” are commonly used for evaluations
- Reflects the grading scale used in schools
- Is pedagogically correct (lowest score for lowest value)
- Provides the opportunity to calculate the average score
- By using a scale where the highest value is a doubledigit number there is no natural midpoint in the scale. The respondent is thus “forced” to choose an answer from either the “negative” (1 – 3) or the “positive” (4 – 6) half of the scale.
The scale must be balanced
When you use a scale as the response option you must make sure that the scale is balanced. Meaning that you should have an equal number of negative and positive values in the scale. The simplest way of resolving this is to use a definition for the outermost values in the scale. If you use “to a very small degree” as the outermost point on the one side, the outermost point on the other side should be “to a very large degree”.
- Avoid leading question text formation in the scale
- In order to avoid the text formation in the question from being leading, you should include both positive and negative values in the question. For example, “How much do you agree or disagree with the following statements…”.
- Use the same type of scale thoughout the survey
We recommend that you make a conscious choice on whether you will use a 5, 6 or 7-point scale. The average from a 5-point scale will not necessarily be comparable to the average from a 6-point scale, etc. At the same time, you avoid confusing the respondent with different scales and meanings.
Matrix questions
Shorten the questionnaire by using matrix questions If you are asking several questions where the same response options will apply, it might be practical to put them into a matrix. This is practical because it shortens the length of the questionnaire, and will create better flow in answering the survey.
- Limit the number of questions in a matrix
- Respondents get tired quickly if there are too many questions to think about at once. The consequence of having too many questions in a matrix might be that the answers are given more arbitrarily the further along in the matrix you come. Based on this, you should have a maximum of 8 – 10 questions/statements/qualities in a matrix.
- Random rotation/shuffling of response options
In some types of questions the respondent has to choose one or more responses from a list. Experience shows that the response options in the upper half of the list are chosen more frequently than those in the bottom half, because the choice is made before reading through the whole list.
