Life Is Not A Randomized Double Blind Controlled Clinical Trial

Or  what is wrong with placebo controlled clinical trials and transparency?

I was in an imaginary meeting with a bunch of Internists and Psychiatrists.  We were debating the available and meager literature on the use of trazodone for sleep.  We got into one of my favorite topics - the double blind placebo controlled trial and that catch all "Evidence based medicine."  The dialogue went something like this:

Internist:  "The evidence from double blind placebo controlled trials is weak for trazodone....."

Me:  "Do I need a double blind placebo controlled trial to tell me to prescribe trazodone for sleep?"

Internist: "Are you saying you don't need evidence.  Oh wait, I was at a conference where the head of the society stood up and and gave evidence that clinical trials have their limitations.  Like they are subsequently proven to not be true...."

Me:  "That is not what I am talking about.  I am talking about all of the hype out there that only 40% of people recover from antidepressants or that they are no better than placebo........"

Internist: "But you do have the algorithmic approaches that show...."

Me:  "Yada, yada, yada - for every algorithm, there is somebody with a meta-analysis to show it is wrong.  No I am talking about the ridiculous notion that psychiatrists could stay in business or want to stay in business if only 40% of the people they treat got better.  If that was true I would have become a recluse 20 years ago and just walked away."

General laughter

Internist: "I would kill for a 40% response rate for some of the problems that I treat."

And so on........

This informal conversation among colleagues is illustrative of the recent arguments that focus on physicians and generally psychiatrists more than other physicians that treatments are ineffective.  They are based on an oversimplified view of placebo controlled clinical trials and conflict of interest.  Many times there are no clear points of demarcation between what is a scientific issue (the technical aspects of the clinical trial) and the ethical issue (conflict of interest issues that may compromise the scientific results).   You can read all about current technical problems with clinical trials, like the common observation that there is a lack of generalizability to clinical populations compared with the highly selected and trial sample that in the case of psychiatry generally has much milder forms of the illness being studied.  Here are a few of the concerns that I don't see being discussed anywhere, especially when it comes to clinical trials of psychiatric interventions:

1.  The crude state of clinical trials:  Clinical trials in psychiatry are tremendously unsophisticated. The trial result generally depends on rating scale or clinician global rating scale results that grossly oversimplify the condition and measure parameters that are irrelevant in clinical settings.  The best example I can think of is depression rating scales that list DSM criteria for depression and then apply a Likert dimension to those symptoms.  In clinical practice it is common to see hundreds of patients with the same score on this scale who have a full spectrum of disability from absolutely none to totally disabled.  Which population might be more likely to exhibit an antidepressant effect?  It is also common to see medically ill patients with insomnia who may score as mildly to moderately depressed who are physically ill and not depressed at all.   The same problems exist with clinical trials of schizophrenia, anxiety and Alzheimer's disease.   When this weak technology is combined with a selection process that eliminates clinical populations with the most severe illness, it should not be surprising that any treatment being studied has a weak effect.

2.  A weak clinical trials data base:  One of the clarion calls of so-called evidence based medicine is the Cochrane Collaboration.  I have looked up hundreds of their reviews and the majority of those reviews of both medical and psychiatric studies is: "insufficient results and methodologically unsound".  I suppose it is good to have somebody make that statement, but if you have proclaimed that you are an evidence based physician - you have nothing to go on.  In fact, at some point you stop going to Cochrane because the results are fairly predictable.   Even in the case where you have a result does it apply to the patient you are seeing?  I don't see many Cochrane studies depressed patients that have right bundle branch block, ventricular premature contractions, or complex atrial fibrillation - all common patients seen in clinical practice.  How many more research papers do I have to read that conclude that "more research is needed" while continued inadequate trials are being published and analyzed by Cochrane?  From the current trends and political correctness of evidence based medicine this will go on forever.  The practical aspects are the very large costs of trials.  That is the real reason that pharmaceutical companies (Big Pharma) sponsor these trials.  The political system in the US has decided to farm out clinical trials to private for-profit companies in the exact manner that the US government has farmed out management of the entire health care system.  In these systems Big Pharma absorbs a disproportionate amount of criticism relative to managed care companies.  Managed care has equated "medically necessary" care with care that can be provided in the form of a pill.

3.  The politicalization of clinical trials:  No clinical trials of psychiatric medications can be done these days without an eye to the politically relevant dimensions.  A new antidepressant needs to get as many FDA approved indications as possible in order to beat the political restrictions of utilization review by managed care companies.   That would include indications not only for depression, anxiety, panic, and social anxiety but also chronic pain and possibly attention deficit-hyperactivity disorder (ADHD).  The best way to beat utilization review is to have the only FDA approved indication in the class.  That is also applied to atypical antipsychotics and that fact seems to escape the critics who focus on the number of prescriptions and what that implies.  Physicians are pawns in a game when there are no suitable tolerated medications for bipolar disorder depression and there are atypical antipsychotics for that indication.  These current political factors in clinical trials preclude a focus on cognition, functional capacity, and endophenotypes - all potentially much more valuable than a focus on diagnostic criteria or rating scales based on those criteria.

4.  A characterization that the average physician is ignorant - at best:  Any political movement is associated with ugly rhetoric.   There has clearly been an effort to stampede any physicians with reservations about evidence based medicine into a Neanderthal category.  The irony is that the criticism often comes from sources like managed care companies, medical certification authorities and the press (bloggers) - all entities with their own high levels of conflict of interest.  Common rhetoric used against psychiatrists has been: "You just don't want to be measured".  If the criticism originates from a government, managed care company or administrative authority there is often the attached explicit threat: "Those  days are over!".  The obvious implicit observation is that medicine and psychiatry is now being run by people who don't know anything at all about medicine and there is plenty of evidence on this blog to back that up.

Many critics seem to get a lot of mileage out of the position that they seem to be particularly anointed to criticize the field.   That seems especially true if they happen to be a physician as in: "My colleagues certainly seem to be a bunch of chumps and therfore can be rejected on a wholesale basis or of course listen to me for enlightenment".  I have never witnessed that level of incompetence in any group of physicians where I have practiced across multiple settings.

5.  The use of statistics for rhetorical purposes:  At this stage after reading research for that past 35 years, I am convinced that you can come up with a meta-analysis that will show whatever you want it to show.  Several years ago in the New England Journal of Medicine there was a study that looked at how well meta-analyses predict the results of available large scale clinical trials.  That study showed 2/3 times.  It is common to see a result and then see a meta-analysis that "disproves" the clinical trials result.  Nobody seems very interested in looking in detail at how sound that meta-analysis was.

Today we have a large number of questionably valuable clinical indicators or quality care markers that are often based on the political rhetoric of the government and managed care organizations.   They may be invalid on the face of it, but there is a second equally important aspect.   These same organizations have no valid approach to looking at the statistics of longitudinal data.  They will look at clinic results or even results from the same physician and convert them all to a numerator and a denominator.  Whether that fraction means anything or not is anybody's guess, but there is an administrator somewhere who will be happy to tell you all about it.

6.  False assumptions about the expertise of physicians:  Much of the rhetoric above is focused on physicians.   Psychiatrists as the most politically disenfranchised group are a frequent target.  In the past years we have endured the ideas that medical treatments being prescribed are ineffective and overprescribed.  That brings us full circle to the opening imaginary conversation.  Physicians are trained to focus on several goals including acute treatment, treatment of chronic problems, and providing care for the dying.  The psychiatrists and physicians that I have had the pleasure of working with have been highly effective is achieving those goals.  When I look at the modest results of poorly designed clinical trials all I can do is shake my head.  I would have quit a long time ago if my diagnostic or treatment capabilities were accurately described in clinical trials and most physicians would have.  Informed clinical treatment is a series of often rapid changes in course, rejecting poorly tolerated treatments and looking for things that work better along the way.  I can change the course of treatment depicted in a clinical trial in one day.  In the trial that result is carried to the end as a failure.  How is it that a clinical trial or this evidence would predict my treatment results by mean?  If a treatment is ineffective or not tolerated in my practice, I don't stop treatment and call that person an unsuccessful intent-to-treat subject.  I work with them to establish effective treatment - often in the same time frame as a clinical trial.  Is it quite literally absurd to suggest that clinical trials accurately describe what will happen in clinical practice and yet that is the state of discourse.

7.  The false promise of transparency:  Anyone who followed politics knows that disclosing potential conflicts of interest  is meaningless in the case of politicians.  The general idea is that politicians would refrain from either accepting money from sources where they are involved politically or just not be involved in that area of legislation.  In reality that does not happen.  Sometimes the politicians involved will speak out against any suggestion that there is a connection between the money they receive and their legislative record even when their activities are consistent with a financial conflict of interest.  The sunlight of transparency that many of the critics talk about simply legitimizes ongoing involvement in areas of potential conflict of interest.   Disclosure is a stamp of approval for involvement.  All of that can be researched on Congressional watchdog sites.   The new CMS web site that posts payments to doctors is hyped as a way to appease all of the critics who seem clueless about transparency.  For an eye opener take a look at their decision point on conflict of interest.  CMS seems much more charitable than the typical blogger, politician or journalist with this disclaimer:

"Sharing information about financial relationships alone is not enough to decide whether they’re beneficial or improper.  Just because there are financial ties doesn’t mean that anyone is doing anything wrong.  Transparency will shed light on the nature and extent of these financial relationships and will hopefully discourage the development of inappropriate relationships.  Given the complexity of disclosure and the importance of discouraging inappropriate relationships without harming beneficial ones, CMS has worked closely with stakeholders to better understand the current scope of the interactions between physicians, teaching hospitals, and industry manufacturers."

8.  A lack of appreciation of the regulatory environment:  In the rush to condemn Big Pharma and anyone associated with them, critics often have an idealized version of regulation in their minds.  If only they had access to all of the clinical trials data to analyze for themselves.  They would personally be able to hold Big Pharma's feet to the fire and only allow drugs to be approved that they deemed to be safe and effective.   They are smarter and more ethical than anyone doing the actual trials and certainly smarter than the regulators.  This is at the minimum a failure to recognize that pharmaceutical regulation is after all a political process.  Like all politically directed regulation there are broad goals of safety and efficacy but they are relative and the regulatory mandate takes that into account.  On this blog I have pointed out several cases of medications that not only I,  but the Scientific Committee employed by the FDA recommended against based on safety considerations, but that were approved by the FDA.   Many drugs that I approved as a member of a Pharmacy and Therapeutics Committee (P&T) were expensive but had minimal efficacy.  We approved it based on political considerations (small but vocal advocacy group and untreatable illness) rather than pure efficacy or safety considerations.  The regulatory environment is currently designed to get promising agents into the hands of clinicians for clinical use.  The limited exposure of patients in clinical trials means that rare but serious side effects will not be recognized until post marketing surveillance occurs.  Every person taking an FDA approved medication should realize that.  The regulatory process is not designed to provide perfect medications because there are none.

There is a lot more to say about this subject like a detailed analysis of how the politicalized version about how physicians work and think has nothing to do with the way physicians actually work and think.  But for today I am stopping here.

Life is not a randomized double blind controlled clinical trial and that is generally a good thing.

George Dawson, MD, DFAPA