STAR*D Results Re-Analyzed, But…

At the end of August, a small journal by the name of Psychotherapy and Psychosomatics published an article that sought to re-analyze the data published in the groundbreaking $35 million STAR*D clinical study. The new analysis suggested that the STAR*D researchers weren’t as forthcoming about some of their results and methodological choices as they should have been. This led to an overstatement of their results, according to the new study’s researchers.

STAR*D was a significant antidepressant milestone study, because it was government-funded (not industry-funded), had a very large sample size, followed patients for a year, and was led by a team of stellar researchers. In other words, it is the gold standard as far as studies go.

If STAR*D couldn’t find much significance for antidepressants, than what hope do we have in the treatment of depression? Is it truly all just placebo effect?

I won’t speak much to the re-analysis done, because in my reading of the new study, it appears that the researchers spent a lot of time simply restating the data in the STAR*D study. Some might say the original STAR*D researchers intentionally obscured some of their data so that nobody would notice how poorly antidepressants eventually fared.

There are two serious charges in the new research against the STAR*D study. One is that the researchers pulled one of the oldest tricks in the research book. They changed how they were going to measure depression and remission, and instead of sticking to using a standard depression measure, they created their own and used that one to define things like remission in their study. That’s a pretty big change, and one that could easily be used to manipulate study results (because your new homegrown depression measure is unlikely to be as robust or time-tested as existing depression measures).

The second charge is that people who weren’t really all that depressed in the first place — the 607 patients who had a baseline HRSD score of 14 or less — were included in overall analysis. They likely shouldn’t have been, since this was a study that was focused on the treatment of moderate to severe depression — not mild depression.

All of which is illuminating and interesting. I’m in general agreement with these two concerns, if indeed they occurred as the authors of the new study contend.

But by the end of reading the new study (Pigott et al., 2010) and an ostensibly objective Medscape news article about the study, something didn’t smell right.

Perhaps it was with the ending paragraph in the study:

Despite the pervasive belief regarding the effectiveness of antidepressants and cognitive therapy (CT) among physicians and society at large, STAR*D shows that antidepressants and CT fail to result in sustained positive effects for the majority of people who receive them. STAR*D authors noted at the outset of the study that the ‘results should have substantial public health and scientific significance’.

As healthcare professionals and in line with what STAR*D authors themselves recommend, we should take notice of what this largest antidepressant effectiveness trial ever conducted is telling us and reassess the role of antidepressant medications and CT in the evidence-based treatment for depression.

The lead author of the study is a psychologist. I found it odd that a psychologist was making statements such as this, based upon a study that barely even examined the effectiveness of cognitive therapy. Why implicate cognitive therapy in this manner, when STAR*D was primarily an antidepressant study? Odd.

Every Study Has a Bias

Ed Pigott, PhD is the lead author of the new study and is one of the owners of NeuroAdvantage LLC. One of the other authors of the study, Greg Alter, is also a psychologist and he’s also an owner of NeuroAdvantage. What, you may ask, is NeuroAdvantage?

NeuroAdvantage manufactures light and sound neurotherapy (LSN) equipment. LSN is a form of biofeedback, and has a fairly good albeit largely small research base (e.g., lots of non-controlled studies with less than 30 subjects). The equipment looks like sunglasses, a pair of ordinary headphones, and a small box that contains the programming. The sunglasses actually have small lights attached to the inside of them. LSN is conducted by flashing lights and sounds in certain sequences, that the wearer then experiences. It lists for $995 on the NeuroAdvantage website. (We found a similar machine that lists for only $239, FYI.)

There’s nothing wrong with LSN therapy. Although not widely well-known amongst most therapists or physicians, it’s generally a safe alternative form of treatment that may work for some people, but not for others (just like antidepressants and psychotherapy).

But there is something wrong with promoting LSN to treat specific diseases or disorders. The U.S. Food and Drug Administration (FDA) doesn’t look kindly at firms who suggest that a specific technology or medical device — as one could argue LSN devices are — can treat specific diseases or disorders without meeting the FDA’s standard of proof. As the FDA notes on their website:

Under the FDCA, a company cannot claim that its drug can treat or mitigate a disease, such as nicotine addiction, unless the drug’s safety and effectiveness have been proven.

So how can NeuroAdvantage, with an unregulated medical device, then make the claim that it Helps to Decrease Symptoms of Depression & Anxiety? The FDA says you can’t claim your device treats or mitigates a disease, like depression or anxiety. Seems like NeuroAdvantage is crossing the line there, and surely two psychologists know this, no?

In fact, it seems clear they know this, because the NeuroAdvantage website is filled with research references, suggesting the effectiveness of LSN in general (but none on their specific devices). They try and overwhelm you with the research references and pretty data graphs so that you might overlook the fact that these devices are not FDA approved. And since they are not FDA approved, they should not be making specific health claims about improving specific mental health conditions.

Tying it to Antidepressants

Despite the authors’ obvious bias here that benefits them personally by noting flaws in others kinds of treatments commonly prescribed to treat depression, I think the gist of their findings about STAR*D are legitimate. They should be answered by a formal reply from the STAR*D researchers in a future journal article, in order to more fully explain the discrepancies these researchers (and others) have noted.

It’s unfortunate that two of the co-authors of the new study are promoting a form of treatment that couldn’t meet the same standards they set for antidepressants and cognitive behavioral therapy. It’s even more unfortunate that they have a direct financial benefit in such promotion, because it makes them no different than any pharmaceutically-funded researcher. It means their message is a little more cloudy and not as clean and simple as it first appears.