Constructing rating scales

Rating scales are simply a set of answers to a question that allow respondents to provide a rating. An example is shown below:

How would you rate the supermarket you use most often in terms of value for money?

Very good
Quite good
Not very good
Not at all good

When constructing rating scales there are some key factors to consider:

1. Should a “Not sure” answer be included?

In the example above I haven't included a “Not sure” option. In this instance respondents are being asked to rate the supermarket they use most often. Assuming they have already been asked their most often used supermarket they should have an opinion. Not including a “Not sure” takes away the easy option and forces respondents to think and make a judgement.

However, if we were to ask for respondents’ perceptions of value for money for a specific supermarket chain then we should include a “Not sure” option. For example:

How would you rate Aldi in terms of value for money?

Very good
Quite good
Not very good
Not at all good
Not sure

The “Not sure” option is a valid choice here as some respondents may have had little experience of Aldi and not feel well enough informed to give an opinion.

2. How many answers should I include in my rating scale?

The number of answers you should include in a rating scale is the subject of academic debate. There is no definitive right answer. The most used options are 4,5 or 7 point scales. Scales with fewer than 4 answers are unlikely to cover all feasible responses while scales with more than 7 answers are unwieldy and complicated.

Irrespective of the number of answers you use your scale should always be balanced. The number of positive answers should always equal the number of negative answers. Rating scales with an odd number of answers include a neutral mid-point as shown below:

To what extent do you agree or disagree with the following statement:

Samsung make great mobile phones

Agree a lot
Agree a little
Neither agree nor disagree
Disagree a little
Disagree a lot

I only consider using 4 or 5 point scales in surveys. My personal view is that 7 point scales are less intuitive for respondents and therefore more difficult for them to understand. It is also harder for researchers to come up with labels for each point that make sense. For example, I've seen rating scales like the one below:

Agree strongly
Agree
Agree somewhat
Neither agree nor disagree
Disagree somewhat
Disagree
Disagree strongly

There are 2 major problems with this scale:

  • Labelling points “agree” or “disagree”. This doesn’t make any sense when there are other agree or disagree options. For example, if someone agrees strongly with a statement then they also agree with it so the answers aren’t exclusive – both answers are feasible. The scale is therefore confusing for respondents.
  • The word “somewhat”. This is a personal bugbear of mine. Few people use the word “somewhat” in real life. Questionnaires should be conversational and use everyday language.

Having disregarded 7 point scales the one decision I have to make is whether or not to include a neutral mid-point i.e. do I use a 4 or 5 point scale? I treat neutral mid-points in the same way as the “Not sure” option in that I will leave them out wherever possible. They also provide respondents with an easy option where they don’t have to give an opinion. However, there are many cases where a neutral option does need to be offered, typically when the respondent might reasonably not have enough experience or interest in the issue to have an opinion. Respondents shouldn't be forced to give an opinion when it is possible they just don't have a view on the subject.

3. Should I use words or numbers?

Some researchers use numeric rather than worded scales. An alternative way to ask for ratings about value for money of the supermarket you use most often is shown below:

How would you rate the value for money provided by the supermarket you use most often on a scale of 1 to 10 where 10 is excellent and 1 not at all good?

1
2
3
4
5
6
7
8
9
10

Asking respondents to give a mark out of 10 is a popular way to ask questions. The ability to report a mean score out of 10 seems to have widespread appeal. However, my preference is to always use shorter worded scales wherever possible.

The question wording for numeric rating questions tends to be more complicated than questions with worded ratings as the answer scale has to be explained. The range of answers is also wider forcing respondents to choose from more options making the task harder.

Also, as each point doesn’t have a worded label respondents interpret the scales in different ways. One respondent may see a mark of 6 out of 10 as quite good while another sees 7 out of 10 as quite good. Resulting data is therefore more difficult to interpret than that derived from worded scales.

Related posts:

Writing attitude statements - a checklist

Balancing scales

"Not sure" vs. "Don't know"

Submit comment

Recent Posts

Twitter