Well, the nation’s biggest research project is finally here. By now you should have all received a large purple and white envelope containing your own copy of the rather weighty 2011 Census questionnaire.
Although its relevance has been heavily criticised this year, the Census should still be regarded as an important piece of history in the making. And let’s not forget that if it wasn’t for a census then the start of the New Testament would be a lot less exciting (boy is born in house, Wise Men find with street map, Herod knocks round but can’t get past over eager well-wishers).
Still, for a variety of reasons ranging from moralistic to suspicion to inertia, many of us won’t bother completing the Census this year. And many of those who do will, intentionally or otherwise, make a bit of a mess of it. These issues aren’t unique to the Census, rather they are just some of problems with self completion research in general.
Lower response rates
The first problem with self completion research is that you never get as many responses as you’d like. Even for the Census, which has a £1,000 fine for non-completion, it was estimated that in 2001 that only 94% of the population actually filled it in.
As any B2B researcher knows, lower response rates than expected means you end up having to employ staff to chase people up. For the 2001 Census, a whopping 81,000 people were recruited to chase up forms which, of course, inevitably impacts on the project costs and lead times.
But even if you get a satisfactory return rate you still have the problem that once you hand a questionnaire over to someone to complete (be it on paper or online) you are surrendering control of the document and there’s no guarantee that the data you get back is what you envisaged.
Open to misinterpretation
Firstly, questions can be misunderstood and therefore answered incorrectly. On the census questionnaire the ‘number of rooms’ question is certainly up for misinterpretation, and will everyone read correctly that bathrooms are not to be included?
The religion question is a minefield; if I am christened but not going to church am I of ‘no religion’? At least if you are completing a questionnaire in the presence of a professional market researcher then these queries can be explained to you. But with a self completion questionnaire respondents normally have to hazard a guess where there is uncertainty.
Rogue responses
A researcher will also ensure all questions are answered seriously and not tongue-in-cheek, as can be the case with self completion. In the now infamous 2001 Census, 390,000 people listed their religion as Jedi Knight. Although this is humorous on the face of it, such rogue responses do undermine the fundamental purpose of the Census which is to provide an accurate snapshot of the population and its characteristics.
A researcher will look to keep the respondent interested and engaged, and make sure each answer is given reasonable thought. This can’t be done for a self completion questionnaire, although Q17 in the Census “This question is intentionally left blank – Go to 18’ did keep me on my toes.
The researcher’s role is also to ensure that the correct person completes the questionnaire so that the right information can be collected. What is to stop someone handing the Census over to their 11 year old child because they’d rather be doing something else? Only trust.
Database accuracy
Finally, you have to be sure with self completion questionnaires that your mailing database is accurate and up-to-date as this will affect your response rate. In 2001, it transpired the Manchester Census database was two years old and therefore anyone with a newer property was not included in the data.
This year, the Office for National Statistics has presumably adopted a ‘better to be safe than sorry’ approach which would explain why a questionnaire was sent to a car parking ticketing machine in Hampshire!
Self completion is still seen as the least intrusive and most anonymous way of being surveyed. And it allows participants to respond at their own leisure. However because of the problems listed, UK research companies often persuade clients away from using self completion questionnaires as a research method. Perhaps this explains why the Government has selected a US arms manufacturer to collect and process the 2011 Census data …
John Gibbons is a market research manager at The Murray Consultancy in Liverpool.
