Suicide in Search Engines? Investigator Bias in the BMJ

You’d think the sky was falling by the way some news media are reporting on a study published recently by the British Medical Journal about online suicide searches:

People searching the web for information on suicide are more likely to find sites encouraging the act than offering support, a study says.

Researchers used four search engines to look for suicide-related sites, the British Medical Journal said.

The three most frequently occurring sites were all pro-suicide, prompting researchers to call for anti-suicide web pages to be prioritised.

Well, yes, according to the BBC. But then you have to look at the actual study to see if what they say actually matches with what the researchers did. Which apparently few journalists nowadays bother to do…

First, the purpose of the study, was to find sites that offer methods of suicide (not, as the BBC purports, to examine what people who are suicidal might find):

We sought to replicate the results of a typical search that might be undertaken by a person seeking information about methods of suicide.

Note the important bias already introduced by the researchers — the people they are pretending to be are people looking for methods to harm themselves. Not people looking for help with suicide. Not people looking for an intervention to stop a suicide. Not people looking for a crisis telephone number to call.

The researchers specifically imagined that most suicidal people are simply looking for methods to kill themselves. I would suggest this is faulty premise to on which to base a study of this nature.

Not surprising, then, these are the search terms the researchers came up with:

(a) suicide; (b) suicide methods; (c) suicide sure methods; (d) most effective methods of suicide; (e) methods of suicide; (f) ways to commit suicide; (g) how to commit suicide; (h) how to kill yourself; (i) easy suicide methods; (j) best suicide methods; (k) pain-free suicide, and (l) quick suicide.

See some pattern there? Well, yeah, virtually every search term mentions methods or ways to kill oneself. Not one mention of support in there. Not one mention of help, treatment, stopping, or crisis. Wow.

The only objective, non-biased term in the list is the plain “suicide” keyword.

So why are the researchers surprised when the search engines, doing their job, returned results that emphasized methods for undertaking a suicide?

Just under half of the 480 web pages visited provided some information about methods of suicide. Almost all dedicated suicide and factual information sites provided such information but, notably, a fifth (21%) of support or prevention sites, over half (55%) of academic or policy sites, and all news reports of suicides also provided information about methods.

Well, yes — that is exactly what one might expect to find given the specific search terms entered!

We replicated the study with only the unbiased term — suicide — and examined the first 10 results in each of the same four search engines. There were exactly zero pro-suicide websites in our findings. Zero. Which is not surprising, because we didn’t stack the deck with 9 other search terms that would tend to favor “pro suicide” websites. (In fact, the number one or two result was one of the most popular suicide pages on the Internet, Suicide: Read This First article written by Martha Ainsworth and hosted by us.)

I’m amazed that the prestigious BMJ would publish such a poor quality and so obviously biased study. If you ask biased questions to begin, you should not be at all surprised that the results exactly match your expectations.

Read the article at the BBC: Fears over pro-suicide web pages