Dr. Google is Calling & You can (Mostly) Trust Her

Nowadays, nearly everybody turns to Google (and to a lesser extent, Bing or a Bing-powered website like Yahoo) to search for information. And nowhere is that more true than when we want to learn about a health or mental health concern.

It also, however, makes you wonder… Google seems to do a pretty good job in giving us relevant results for all kinds of information. But how’s it do with mental health information results specifically? Are the results you get from Google and Bing when you conduct a search for mental health information of high content quality and useful?

So last year, I ran a research study (which was just recently published) to find out.

The study (Grohol et al., 2013) looked at 440 web search results from the two major search engines, Google and Bing. We typed in 11 of the most commonly-searched for mental health terms (using Google Trends and Bing’s webmaster tools to determine what they might be), and looked at the first 20 results (because more than 95 percent of people stop looking after page 2).

My colleagues Joe Slimowicz and Rebecca Granda then examined each of those 440 web pages and rated them on a number of content measures, and also looked at readability and other characteristics of each site (like whether they displayed the HONcode or not). ((I was not involved directly in the ratings of websites since some of my own resources here at Psych Central would likely be rated in the study.)) We used common and reliable web quality analysis tools, such as the UK DISCERN instrument, as well as an adaptation of the Depression Website Content Checklist, which we adapted for each of the 11 mental health terms.

What we found should put your mind at ease — most of the websites we examined had pretty good quality:

Of the total Web sites analyzed, 67.5% had good or better quality content.

Nearly one-third of the search results produced Web sites from three entities: WebMD, Wikipedia, and the Mayo Clinic.

If find it interesting that a good chunk of the mental health information Dr. Google returns to you is coming from just three entities (out of the thousands of possible websites that have this kind of information on it). WebMD, which came up first with 15.5 percent of the search results, does something that, in my opinion, is a little sneaky. WebMD runs two other major websites not branded as “WebMD” — Medicinenet.com and eMedicineHealth.com — but run and managed by the same company. ((It’s not clear why Dr. Google allows this sort of near-duplicate content, as WebMD is the only company I’m aware of that can get away with this.)) Wikipedia came in second at 8.6 percent of the search results on mental health topics, and the Mayo Clinic came up with 7 percent.

Sadly, mental health websites targeted at helping people get the most detailed, accurate and in-depth information didn’t do as well. Little ‘ole us (Psych Central) has only 4.3 percent of the search results, while the stellar HelpGuide.org had 4.1 percent of the results. This is likely because those big websites like WebMD and Wikipedia have huge search footprints and inbound links — which impact their search results positively across the board.

Disturbingly, 3.9 percent of the search results were to only an online dictionary definition of the term we were searching for. I say “disturbingly,” because it is rare for a person to be searching for health or mental health information online only looking for a 2 sentence dictionary definition.

Contrary to some people’s fears, it was obvious to tell who was running a website in nearly 94 percent of the search results. While most websites had a privacy policy (89.1 percent), only a minority had an editorial board (39.3 percent).

Readability remains an ongoing issue for health and mental health information online. This may be a result of the fact that health and mental health information can’t help but use technical jargon sometimes, in order to accurately describe a symptom or condition. We found that the mean Flesch Reading Ease score was 41.21, and the mean Flesch — Kincaid Grade Level score was 11.68. Ideally, we’re told we should be writing for an 8th grade reading level, but most of the mental health information we found was written closer to a 12th grade reading level.

Much is made about the presence of absence of certain badges on websites, perhaps as a stand-in for determining quality on-the-fly. We only looked at the HONcode badge, because it is free to display and has been around nearly as long as health information has been on the web. The presence of the HONCode badge (and, separately, noncommercial status) was found to have only a very small correlation with Web site quality. So small, in fact, that I wouldn’t be comfortable using either characteristic in my day-to-day online travels. (Web sites displaying the HONCode badge and commercial websites also had lower readability scores.)

I’ll let the paper sum up our findings:

The present study confirms that the use of a single Web site characteristic — such as the display of the HONcode badge or commercial status — is unlikely to be a reliable indicator alone of Web site quality, and that readability remains a challenging issue for mental health Web sites.

These findings taken together as a whole suggest that search engines’ algorithms for returning relevant, good quality search results can largely be trusted in this specific health content domain.

Reference

Grohol, JM, Slimowicz, J, Granda, R. (2013). The Quality of Mental Health Information Commonly Searched For on the Internet. Cyberpsychology, Behavior & Social Networking. DOI: 10.1089/cyber.2013.0258