Surveys Room 101: My pet hates about surveys

As an experienced survey designer I often see questionnaires that frustrate me in some way, be they too long, too complicated or just designed without any regard for the respondents who are asked to complete them. I try not to let these frustrations affect my mood. However, there are some relatively minor, but recurring, issues that anger me to the point where I’m shouting at my PC screen and pulling out the few bits of hair I've got left.

Here are some of the gripes I’d like to banish to George Orwell’s fictional Room 101:


1. The use of gender icons


Toilet Door icons

What purpose do these toilet door graphics serve in an online questionnaire? I know the difference between genders without needed to be reminded.

And one more thing, what is the text "Please select one option" there for?


2. Being asked the same questions again and again and again




You join a panel and as part of the enrollment process you fill in a questionnaire that asks for some details about yourself – gender, age, where you live and so on. They tell you they need these details so they can send you surveys that match your profile. You're then sent surveys and every one of them starts by asking the same questions about gender, age etc. It’s like they're being programmed by someone with the memory of a goldfish. My gender doesn’t change and my age changes on the same day every year. There’s no need to ask me twice!


3. Prompted awareness questions with long lists of brands everyone has heard of


Long well known brand list

The question above has an answer list that contains 14 different brands, all of which I imagine most people in the UK have heard of. Answering truthfully, I had to check all 14 boxes to get to the next question. I'd even told them I'd heard of Sony, LG and Apple at the previous spontaneous awareness question and still I had to tick the boxes to let them know I hadn't forgotten about the brands between questions.


4. The word "somewhat"


angry somewhat

Perhaps the least rational of my pet hates, I detest the word “somewhat” in the context of surveys. It is used with annoying regularity with “somewhat agree” and “somewhat disagree” being particular favourites. But who on earth uses the word “somewhat” nowadays? A few direct relatives of the Queen maybe? It’s not in common usage so don’t use it in your questionnaires. Surveys should be conversational, not linguistic relics from a bygone era. What's wrong with "agree a lot" and "agree a little"?

Unfortunately The Guardian have published this entertaining article in praise of the word “somewhat”. It is so persuasive that I even found myself somewhat in agreement.

5. Answer choices that mean the same thing

Take a look at the answer options for this question:7 point scale difficultiesIgnore the clumsy question wording and the text asking the respondent to choose an answer from 1 to 7 when there are no numbers attached to each response. That's annoying but not as frustrating as the lack of differentiation in the answer text. If "I would certainly consider" buying the beverage then, also, "I would consider" buying the beverage. Likewise if "I would quite consider" it (whatever that means) then that means "I would consider". The answers aren't mutually exclusive, they aren't distinct. This means respondents get confused and results are difficult to interpret.

This type of irritation is often seen in agree/disagree sales with, for instance, "agree strongly" and "agree" being part of the same rating scale. If I "agree strongly" I also "agree". Which box do I tick? Make the answers distinct and make it easy for respondents!


So now you've read about my pet peeves, what should be added to the list? We'd love to know what gets on your nerves, puts your back up and drives you against the wall? Please share in the comments below:



Submit comment

Recent Posts