Here’s the challenge faced by all health journalists. If you want to report independently about a study, you probably should have a background or degree in statistics (or some related field that gives you a good foundation or grounding in statistics). Otherwise, you’re basically relegated to rewriting organizations’ and journals’ press releases that report on the findings of a study.
One thing the peer review process really doesn’t do very well is to reign in authors when it comes to the conclusions they make in their Discussion section. The other thing it doesn’t do very well is to ensure the limitations of a study are clearly noted by the author. There are many reasons for these failures, but it generally happens because the author is the expert in the field of their study, while peer reviewers are often not (but generally well-versed within the broader field). So peer reviewers will often not argue with an author’s conclusions if they seem reasonable and based upon the study’s data.
But as the The Worst Science Stories of 2007: STATS Dubious Data Awards shows, relying on the author to be unbiased and report their findings in a similarly unbiased fashion isn’t always the best idea. And if an author may occasionally overstate a finding or piece of data, wait until the public relations department of the journal or their university gets a hold of the study and writes the press release. They will ensure that if there’s anything even remotely, possibly interesting about the study, it will get a lot of attention.
Many health journalists do a great job with all of this, which is an ongoing, constant effort that takes a lot of time, reading, and further research of their own. But as the STATS Dubious Data Awards show, sometimes stories slip through the cracks and get a boatload of media attention, even though they are based upon dubious data (or dubious analysis of good data).
Three stories on relationships and mental health stories that we reported on (without a whole lot of independent reporting, I might add) made their cut:
Single women left at the altar by statistics
Time Out New York sent a wave of panic through the city’s single women in June by reporting in a cover story that there are 185,000 more single women than men looking for love. It turns out, however, that the excess number of single women is due to men dying at younger ages than women. If you look at the male/female numbers in the younger age groups, in most, there are significantly more men. For example, there are 211,590 men aged 18 and 19 in the NY Metro area — but only 201,282 women.Husbandless teens languish at home
The New York Times went further and claimed in a front-page article in January that more women are living in the United States without a husband than with one. The claim could only be supported by counting women between the ages of 15 and 17, 90 percent of whom live at home with their parents.Will one joint make you schizoid?
In July, the Associated Press — and many other news organizations — reported that “Using marijuana seems to increase the chance of becoming psychotic… even infrequent use could raise the small but real risk of this serious mental illness by 40 percent.” Since marijuana use rates have skyrocketed since the 1940’s and 50’s, going from single digit percentages of the population trying it to a peak of some 60 percent of high school seniors trying it in 1979 (stabilizing thereafter at roughly 50 percent of each high school class), we would expect to see this trend have some visible effect on the prevalence of schizophrenia and other psychoses.Roughly one to two percent of the population has schizophrenia (and another two percent or so have other psychotic disorders), and this percentage does not vary much with the region within the U.S. Over time, diagnosis of schizophrenia has changed, making it almost impossible to evaluate whether low-level exposure to pot could increase the risk by as much as 40 percent.
Ironically, schizoid is probably not the right word to describe what the authors intended. Psychosis is a symptom of schizophrenia, or a schizophrenia-related disorder (like schizoaffective disorder). Schizoid refers to a personality disorder that is characterized by a person who has no close relationships, no desire for them, takes little pleasure in life, lacks any close friends, and has little emotion.
