Phase III clinical trials are the final phase of research needed before a drug receives U.S. Food and Drug Administration (FDA) approval. Two fairly large-scale studies are needed and they need to show the drug is both safe and effective on the subjects tested.
There’s been a long-standing problem with such clinical studies, however, one that the FDA has long been aware of but powerless to fix. They are purposely designed to employ stringent inclusion and exclusion criteria that may exclude a substantial portion of the population. In other words, the people the drugs are studied on are not representative of the people that will actually be receiving the drugs once approved.
In other words, Phase III clinical studies are stacked in favor of finding positive results for the medication under study.
A new study published in the latest issue of The American Journal of Psychiatry by Wisniewski and colleagues (2009) decided to put the hypothesis to the test by examining the great data generated by the government-backed STAR*D project. “STAR*D was designed with broad inclusion and minimal exclusion criteria to ensure recruitment of a representative sample of treatment-seeking depressed outpatients who receive treatment in typical clinical settings,” the researchers noted.
The researchers divided the STAR*D subjects into two groups — those who would’ve qualified for a Phase III clinical trial (the “efficacy sample”), and those who would not have:
STAR*D enrolled a total of 4,041 participants, 2,876 of whom made up an analyzable sample (having at least one postbaseline visit and a score of 14 or higher on the HAM-D). Of these, 2,855 could be classified into the efficacy sample (N=635, 22.2%) or the nonefficacy sample (N=2,220, 77.8%)
You can see an interesting phenomenon already based upon the researchers’ classification. Only 22.2 percent of the subjects in the STAR*D would have qualified for a Phase III clinical trial. The vast majority of subjects would not have qualified, which immediately calls into question the generalizability and usefulness of data that would only apply to 22.2 percent of the population. (Previous research has suggested this number may be as low as 9 percent.)
They also found that the efficacy sample, as compared to the non-efficacy sample of depressed people, had:
- Shorter duration of depression
- Lower rates of prior suicide attempts
- Lower rates of family history of substance abuse
- Lower rates of anxiety and other non-depressive symptoms
- More likely to be seen in a psychiatric specialty care setting
- Less likely to have severe side effects
- Less likely to have a serious adverse event (either psychiatric or due to the medication)
All of which may readily explain the observation by most clinicians that medications rarely meet the expectations found and published in peer-reviewed research (the so-called “gold standard”):
[A]ll measures of outcome showed significant but modest differences between the groups, with the efficacy sample having, on average, better outcomes. These differences were consistent in the direction and magnitude of effect when examined separately in primary and psychiatric care settings.
Given these between-group differences, the smaller efficacy sample is clearly not representative of the more inclusive, treatment-seeking population. By inference, a patient sample that meets the inclusion criteria for a phase III clinical trial is not representative of depressed patients seen in typical clinical practice, and phase III trial outcomes may be more optimistic than results obtained in practice.[…]
To our knowledge, the current study is the first to examine the differences in treatment outcome. Notably, response and remission rates were poorer and the times to response and remission were longer in patients ineligible for efficacy trials. Thus, current efficacy trials suggest a more optimistic outcome than is likely in practice, and the duration of adequate treatment suggested by data from efficacy trials may be too short.
There is an obvious trade-off in opening up Phase III clinical trials to a broader and more representative sample of patients — medications will not meet the FDA’s threshold for efficacy, and therefore not be approved. Therefore, unless the FDA were to change their Phase III requirements, this situation is not likely to change on its own, independently, any time soon, despite data such as this that shows the research is fundamentally flawed.
In research, how you choose your sample is a fundamental way that you can help shape your results. Researchers know this, of course, and often will pick inclusion or exclusion criteria for their sample that will lead to the greatest likelihood of them finding significance in their data. Once you know what to look for in sampling (e.g., Is it a randomized or a convenience sample? Are the inclusion/exclusion criteria overly strict? Is it representative of the population and demographics?), you can tell a lot about the actual usefulness and generalizability of the study’s findings.
The latest research continues a long line of similar studies that give us insight into why medications rarely seem to work as well (or as with as few side effects) as their clinical trials indicated.
So if you’re feeling frustrated about your antidepressant or psychiatric medication not working as well as advertised, this may be one of the reasons why — it’s not as effective in the general population as it is on the cherry-picked sample studied.
Reference:
Stephen R. Wisniewski, A. John Rush, Andrew A. Nierenberg, Bradley N. Gaynes, Diane Warden, James F. Luther, Patrick J. McGrath, Philip W. Lavori, Michael E. Thase, Maurizio Fava, and Madhukar H. Trivedi. (2009). Can Phase III Trial Results of Antidepressant Medications Be Generalized to Clinical Practice? A STAR*D Report. Am J Psychiatry, 166(5), 599-607.
6 comments
Hello John,
Chuck and I recently reviewed the literature on medication efficacy for Borderline Personality Disorder. One recent article showed some success in treating symptoms of BPD with medication. However, the studies for the most part excluded people who had substance abuse or recent suicide attempts–two symptoms that are usually present in people with BPD. So, what you are reporting is going on in many different areas of psychopathology.
Another nice post. Makes me wonder why the ‘usual suspects’ at Furious Seasons attack you as being pro-medication when you put posts like this out every few weeks at least.
By the way, an observation I find none of my colleagues have really understood or even considered in the first place, in my opinion, is that the examiners of Star*D cut off Celexa at 60mg, which is equivalent to about 15 to 20 mg max for Lexapro, a cousin of Celexa. And yet, I continue to interact with patients who have been prescribed 20, 30, even one at 40mg of Lexapro. A study like this would have at least maximized the dose of the initial medication, so if they felt 60mg Celexa was a max dose, what the hell are psychiatrists thinking in prescribing the doses of Lexapro these days? Makes you wonder who is defining ceiling limits, because I find the Forest Reps (who make Lexapro, and Celexa prior to it being generic) struggling for a reply when I mention these prescription doses with Lexapro. Is 20mg a ceiling or not?! If you the reader find this a matter of interest, ask questions!!!
By the way, Furious Seasons with a nice group of posts today about the cluelessness of pharma. Sad stuff going on, especially if his comments about the inappropriate role between NAMI and pharma has legitimacy.
“the road to hell is paved with good intentions”. Every Doctor should have that quote above his/her door at the office.
In reply to Dr Laura Smith above, watch out for any literature that promotes meds for Axis II disorders. Reinforces the wrong message, as the only meds value is for co-existing Axis I disorders, and you have to wonder with the mood lability and inflexible thought patterns in cluster B disorders especially, if there is a true Axis I in the first place. Personally, I think medicating Axis II runs more a risk of increasing dysfunction before improving function. Especially with antipsychotics and benzodiazepines!
Just an opininon, of a psych doc.
I was diagnosed with paranoid schizophrenia. I suffered episodes of depresion in high school after my grades started to drop or I earned lowere grades. Prior to this happening I was an honor student and so I took it hard. In occurance I began to suffer from depresion trying to OD on medication. After seeing a theripist my demensia decreased and due to the medication I began to get back on track on daily routines. I was pacing myself and wearing clothes backwards to class, talking out loud screaming under pressure even. I have an understanding family that gave me lots of moral support to get through the dementia. I feel somehow I lost my memory about my past and still have delusional thoughts about the past. Now I am able to sustain a normal life under medication that I will have to take for the rest of my life. I continued with my education and I am determined to graduate with a BS from the university. I realy needed a psychiatrist and theripist to help me with my condition. The paranoia was worse with hearing voices inside my head under stress. Life is god scent.
Can’t say I understand what I am to understand from all of this.
Anyway, just saying that I have been offered to be in two different Clinical Trials. After a couple visits each I ended up no longer qualified. I wasn’t ill enough….
Take a listen to WHYY’s coverage on this issue – here is a link
http://whyy.org/cms/news/health-science/behavioral-health-health-science/2009/04/30/majority-of-people-who-take-anti-depressants-not-represented-in-clinical-trials/7620
This is an interesting article. As a former FDA field investigator conducting inspections of clinical trials I can assure you that inclusion/exclusion criteria in the protocol are considered to be important to review and many Warning Letters reference these inclusion/exclusion criteria. I have attached a link to this article on my blog, Carl’s Blog on FDA Stuff and a few comments of my own. I think we really need to take a hard, scientific look at study design in this country. I’m not sure I entirely agree but there is a lot of food for thought.