It is easy to be confused about the real value of opinion polls and surveys.
There seem to be so many these days, but I think it’s time to blow the whistle on the one-question opinion poll that masquerades as ‘research’.
Once upon a time you might be accosted in the Hay Street mall by an earnest young thing armed with clipboard and pen asking lots of questions, all in the interest of serious research.
Today, we are much more efficient. We have either the phone survey, those annoying people who call just as you’re sitting down to dinner, or the on-line poll.
These are quicker and less painful than the old model for two reasons: the technology is better and they often ask only one question.
But are they accurate? Do they actually reflect what the majority of people really think?
Well, that depends on the subject of the survey and the form of the question.
One-question surveys are pretty accurate about black-and-white issues. Sport, politics and other matters of personal taste are their bread and butter.
But you might as well ask whether beef tastes better than lamb, or whether Fords are better than Holdens.
One-dimensional questions are fine when the issue is more or less immaterial to our overall well-being as individuals or as a society. For these questions, a simple ‘yes’ or ‘no’ is all you need to say.
The danger comes when single-question polls purport to reveal public opinion on very complex issues, matters that really do impact quite profoundly on our health, happiness and well-being.
A simple ‘yes’ or ‘no’ option to one question just doesn’t capture the complexity of some issues, yet often that is all you are allowed to say.
The fact is all reputable research in Australia must be approved by scientific and ethical review panels.
I have reviewed hundreds of research projects for a number of universities over the last 20 years. No serious research is ever conducted on the basis of a one-question opinion poll.
When they want to build up credible data about an important matter, researchers use questionnaires or multiple-question surveys even if they are really investigating only two or three key issues.
By asking more or less the same few questions in several different ways, serious researchers establish a degree of objectivity.
They can work out how well the respondent understands the complex issues, and they can validate responses by cross-checking various different answers.
This technique also prevents the researcher from leading the respondent down a preferred path, which can happen when a question is ‘loaded’.
For a good example of what not to do, here is a single-question survey currently doing the rounds in New South Wales: ‘If a hopelessly ill patient, experiencing un-relievable suffering, with absolutely no chance of recovering, asks for a lethal dose, should a doctor be allowed to provide a lethal dose, or not?’
Can you guess which answer the pollsters want you to give?
Of course they also asked people a lot of demographic questions – their age, gender, income and so on – but there is only one material question to answer.
By any serious standard this single-question opinion poll fails the objectivity test on several grounds.
The odds are stacked against you from the start. Simply by making the answer ‘no’ seem manifestly cruel and unreasonable, the question leads you down a one-way street to an obvious ‘yes’ which is the preferred answer.
By describing the illness as ‘hopeless’ and the suffering as ‘un-relievable’, the question has already determined that you have no other options.
Never mind the difference between physical pain and other kinds of suffering, and forget about the alternatives actually offered by excellent medical care.
Certainly don’t worry about the practicalities of legislating, monitoring and controlling practices which have proved uncontrollable elsewhere in the world.
The sponsors of this poll want you to ignore the fact that some incredibly complex medical, social and personal issues have been reduced to a single black-or-white question that is clearly designed to produce only one outcome.
If there is no objectivity in the question, there will be none in the answer.
The one great truth about surveys and polls is this: the more profound the issue we wish to investigate, the more careful, rigorous and comprehensive must be our investigation.
Single-question opinion polls eliminate complexity by simply ignoring it. That is why they are best suited to inconsequential matters of taste.
They cannot produce the hard data or solid information we need to chart a course through more complex social issues.
– Rev Dr Joe Parkinson is director of the LJ Goody Bioethics Centre, Perth