Ed Silverman over at Pharmalot reports on the media coverage of a new study published by the Journal of Medical Ethics which shows a disturbing trend — more and more journals are retracting journal articles they previously published.
Worse yet, nearly 32 percent of the retracted papers are not noted as retracted. “Retracted” in scientific language means that the paper has been withdrawn and should be ignored — as though it never existed in the scientific literature. Retractions generally occur because of sloppy research and errors in the data calculations, collection or statistics, or because of fraud.
Is this a trend pointing to lower quality research and sloppier methods being employed? Or perhaps that because more people than ever can read the scientific research, more mistakes are being found after publication?
The data are hard to resist. I’ve graphed them out below, with 2011 estimated data based upon the 210 retractions so far this year (through July 2011):
What we see here is that only 10 years ago — modern times by any standard — only 22 retraction notices were published. Out of the thousands of research papers published every year, just 22 were retracted.
For 2011, it’s on course to hit 360. The surge from 2001 to 2006 represents over a 500 percent increase, while the increase from 2006 to 2011 is only 159 percent. These are very scary numbers.
According to the blog article, “After studying 742 papers that were withdrawn from 2000 to 2010, the analysis found that 73.5 percent were retracted simply for error, but 26.6 percent were retracted for fraud.”
But as regular readers of World of Psychology know, we don’t really care about the numbers — not without proper context. After all, the amount of articles published every year has also gone up. So here’s a nice graph that demonstrates how retractions per 100,000 papers published has also skyrocketed in the past decade. Something is definitely amiss.
Retractions due to fraud are also on the rise:
Ivan Oransky, the executive editor of Reuters Health and a co-founder of the Retraction Watch blog that began recently in response to the spate of retractions, writes us that the simple use of eyeballs and software that can detect plagiarism has made it possible to root out bad papers. […]
And why is there more fraud? As the Wall Street Journal notes, there is a lot to be gained — by both researchers and journal editors — to publish influential papers. “The stakes are so high,” The Lancet editor Richard Horton tells the Journal. “A single paper in Lancet and you get your chair and you get your money. It’s your passport to success.”
A few notable retractions include an episode at the Mayo Clinic, where a decade of cancer research – which was partly taxpayer-funded – was undermined after the clinic realized that data about harnessing the immune system to fight cancer had been fabricated. A total of 17 papers published in nine research journals were retracted and one researcher, who maintained innocence, was fired.
Ouch.
All of which is to say that the foundation of our science — peer-review publication — is increasingly suffering from problems that make it tougher and tougher to not take anything read in a journal with a grain of salt.
The scientific process has a solution to this problem, of course — it’s called replication by other, independent researchers of one researcher’s initial findings. However, in this fast-moving world, few people wait for replication any longer, and are quite happy to trumpet findings from tiny studies done on a few undergraduate college students.
The other solution is peer-review. Peer-review is the process where journals vet incoming scientific articles by reviewers (who are usually experts in the topic they are reviewing). Those reviewers are supposed to be able to objectively pass judgment on the research and determine whether it is worthy of publication in the journal the study was submitted to.
But as I’ll write in a future blog entry, the current peer-review process is horribly broken — another one of the possible reasons for the rise in retractions. Until the process is fixed, it’s unlikely to help curb the rise in retractions.
The upshot is simple but disturbing — we can no longer reliably or generally trust research findings published in virtually any journal — medical, psychological, or otherwise — without taking those results with a grain of salt and waiting for replication to confirm the findings.
Read the full entry: Retractions Of Scientific Studies Are Surging
4 comments
“we can no longer reliably or generally trust research findings published in virtually any journal”.
Absolutely. We let the “experts” make decision for us because we think they are more qualified and we trust them to decide what’s best for us. This is the mentality that has to change. Everybody needs to be a good investigator. Don’t trust something just because the source has “credentials”. You have to question MOTIVES and consider HISTORY.
The numbers are alarming, but it is just the tip of an ice-berg. More so is the implication on published/on-going research which are based on these retracted/error-ridden papers. For instance, over the past 3 months, I came across two such papers, one of which was published in an exceptionally respected journal. This has been transformed into a seminal paper of sorts – having over 100 citations (most of which are positive) in four years, some of which considered this as their core paper. However, when one reads the paper twice, certain errors are glaringly evident which topples the entire validity of the interpretation. None of the papers which cited this has noticed this (yet). And there are no retractions. It is quite a scary scenario if this is applicable to a proportion of peer-reviewed published research.
This is very alarming as people depend on what the “experts” are saying. With the “hope” that it is supposed to bring, research plays a very important role in our world and this news changes everything.
If we cant trust the “experts” then who can we trust and look to for accurate information?
I can’t remember who said it, but as it was I heard it said on a PI Institute video that perhaps academia should look outside for new ideas.
Academia I fear has become arrogant and now showing itself as unsofisticated and most un-trustworthy.
There is a new research and theory that is about to become known to the public that presents a radical and profound paradigm shift in the Social Sciences of which I believe Thomas Kuhn said paradigm shift was not appropriate.
We will see in the near future what the truth is.
There is no truth. not in science and not in life. Veritas filia temporis, Giordano Bruno said. The truth is a daughter of the time.
(1) The increase of fraud has something to do with increased pressure in academia. Students who *apply* to a PhD program are expected to have publications! Applicants to assistant professor positions are expected to have 10 and more publications, and as many as possible as sole author, and in prestigious journals on top of that. Meanwhile, more people submit to those same journals and journals have higher rejection rates and/or longer publication times. And I won’t go into the potential biases of the review process, where unfortunately selfish interests, politics, and egos have become more important than the quest for knowledge.
(2) Which brings me to why there is an increase in the *detection* of fraud. Because who says that an increase in retractions really means that there is just more fraud? It is much more likely that there was fraud all along, but that it is getting detected now for reasons that might also have something to do with the increase of pressure. That some of those papers we always thought were conducted properly, and that we have tried to build more research on but failed (some studies could never be replicated but never succeeded in getting published, which is an entirely different problem), that those papers were fabrications after all. We might never know.
As for trust: one should not trust anybody in science (or in life), not even one’s own judgment. I bet some of the coauthors of people like Diederick Stapel thought that they can trust him.
The damage is greater than can be imagined: damage to the scientific community, to its public appearance, but even worse, damage to the knowledge base we have. How am I supposed to delete all these fraudulent, retracted studies that have formed my thinking from my knowledge base?
Veritas filia temporis. We have to start all over again.
Comments are closed.