In their book Super Freakonomics, Stephen Levitt and Stephen Dubner discuss a common problem facing hospitals, doctors not washing their hands as often as they should. In 1999 the Institute of Medicine estimated that between 44,000 and 98,000 Americans a year die from avoidable hospital errors. The leading cause of these errors was wound infections, easily prevented by doctors washing their hands. The doctors knew that not washing their hands put patients at risk. Awareness wasn’t a problem but, despite knowing the potential consequences, doctors still weren’t washing their hands as often as they should. Various schemes were tried including posters, email messages, wall-mounted disinfectant dispensers and even handing out $10 Starbucks cards to doctors seen washing their hands properly. None of them shifted behaviour significantly. However, an effective solution was found in one hospital, the introduction of screensavers on all computers showing pictures of grisly bacteria-laden handprints. Hand-hygiene compliance shot up.
Like the doctors of the past the vast majority of survey designers are guilty of burying their heads in the sand and ignoring the consequences, the real threat they pose to the long-term viability of survey research. There’s probably not a month goes by when I don’t read a blog by a fellow market research professional that calls for the industry to write shorter, more engaging surveys. Evidence has been produced that shows that response rates fall and drop-out rates rise when questionnaire length goes up. Despite this weight of opinion we don’t seem to be doing anything differently. The highest profile, most reputable market research companies are still sending out the same old gruelling questionnaires. We know it’s wrong but we still do it.
For the hard pressed survey designer there is little incentive to do things differently. Producing shorter, more focussed, more engaging studies requires additional care, time and skill, costs that few are seemingly willing to bear. There are rarely any immediate, personal consequences for bad survey design. Data quality might be compromised, but who notices and who cares? Respondents may be less inclined to respond to future surveys but that’s not a concern for now. In my experience, the gatekeepers to respondents, panel companies and fieldwork agencies have low levels of quality control. Few insist on any particular standards before subjecting respondents to the latest elongated gridfest.
For the perpetrator, bad survey design is a victimless crime. For online surveys the designer doesn’t see or feel the respondents’ pain. For face-to-face and telephone surveys, interviewers bear the brunt of respondents’ ire and boredom not the execs who design the surveys.
Shouldn’t our efforts now be concentrated on how to change behaviour rather than repeatedly highlighting the issue? Awareness alone won't change behaviour. What incentives can change the way survey designers act? What is our equivalent of the dirty hand screensaver? Let’s start the debate.,,