Clinical Trials Versus Clinical Treatment – Key Differences

 

This is always an ongoing source of debate in psychiatry – largely because of the forces that seek to delegitimize the field.  This post is not directed at them since they have a long track record of producing the same biased arguments.  This is my perspective as a person who has successfully treated people with severe psychiatric problems and been an investigator in clinical trials. I have first hand experience with the limitations of clinical trials and they can be considerable.  I am certainly not unique in that regard and it is important to know why those of us with that experience can dismiss headlines about psychiatric treatments not working as well as many of the associated reasons.

Let me preface these comments by saying that running a clinical trial at whatever level you are responsible for is hard work. I have been a co-principal investigator but most of my work was focused on research subject follow-ups and patient safety. I was the guy who got the call about unexpected side effects, lab reports, ECGs, and imaging studies. In those cases, it was my job to decide if the problem was due to the study drug (or placebo) or an intercurrent illness and decide if it was safe for the research subject to proceed.  If the decision was to stop the protocol, there is always at least one person who upset with that decision. There is a study coordinator who wants to maximize retention and study completers. There are the research subjects and their families who may consider the research drug the latest hope – even though they do not know if they are getting placebo. There are other investigators and research assistants who all want to maximize research participation.

Let me start by framing the main differences between research and clinical treatment starting at the top of the diagram.  Clinical trials are sampling the general population on a very specific feature.  In medicine that is typically a disease state. That selection process is modified to be even more stringent to eliminate other variables.  In the diagram this is indicated by the inclusion and exclusion selection criteria. That does not exist in clinical practice where any number of complicating factors need to be addressed in addition to the main indication for treatment. 

This recruitment phase in clinical trials is critical because there is limited funding, and adequate sample size must be recruited, and therefore there is an acceptable time frame. 70% of clinical trials do not recruit the number of subjects they need in the expected time frame.  Currently in the US, less than 3% of adult cancer patients participate in clinical trials (ref 1 p. 268).  In older research it was often stated that only 5% of eligible subjects participate in research. In my experience that number is accurate.  In some of my studies it approached 2% at times.  In other areas the numbers are higher but they typically max out at about 25% and there is a significant difference between experimental trials and retrospective or epidemiological trials that are less demanding.

The ratio of enrolled/screened is important because it suggests a skewed sample from the population that might be more evident if the reasons for declining research were explored.

Response to the recruitment problem is typically that it is taken care of by randomization – but is that accurate?  For example, if people do not want to take a chance on a new medication or randomization does that eliminate potentially good treatment responders from both arms of the trial?  If a trial is debated in the press – like many psychiatric trials are - will that increase the nocebo response in both arms? If a trial involves discontinuation or substitution of a current medication that had been somewhat effective does that lead to increased problems with nocebo effects and drop puts.  The impact on the physician-patient relationship can be affected by both offering and not offering trial participation.  Physicians are most likely not to suggest that stable well treated patients participate for altruistic purposes.  Recruiting pressure on the clinical trial team may lead them to accept suboptimal research subjects with a history of participation in multiple trials, subjects with milder illness or equal comorbidity (e.g. anxiety = depression), or subjects who are more likely to drop out.

These are just a few of the factors that can affect the recruitment and enrollment phase that are not very well investigated or recorded. Speaking from experience, I can say that what looks like an acceptable treatment protocol is often modified as investigators start to panic about low enrollment in the trial.

On the clinical side there are different problems. No matter how complex the patient’s situation there are typically waiting lists of patients to be seen by psychiatrists.  This is true even though only about 6% of the medical workforce that prescribes psychotropic medications are psychiatrists. There are no exclusion criteria and rather than the potential trial participant effect where trial participants are likely to do better even if the medication is ineffective (ref 1, p. 258) – patients referred to see psychiatrists are generally less healthy, more likely to have chronicity, less likely to adhere to treatment, more likely to have a substance use disorder, and more likely to have medical and psychiatric comorbidity.

In addition to the placebo and nocebo effects there are a number of lesser-known factors affecting the experience of people in clinical trials and clinical care.  Some of them will alter the entire trial sample.   For example, a recruitment bias due to failing enrollment can result in equivocal cases being enrolled (for example an adjustment disorder with depressed mood rather than major depression in a depression trial). These subjects are more likely to appear to respond to nonspecific factors.  Many people diagnosed with depression have a long history primary insomnia and may appear recovered if their insomnia is treated by the antidepressant or adjunct medications allowed in some protocols. I have participated in some protocols where all of the investigators were instructed to avoid any semblance of psychotherapy and not enroll anyone who might have a personality disorder.  These are some of the occurrences behind the scenes that can affect clinical trials.

The number of times a person volunteers for clinical research can be an issue. There is very little written about this, but there are some centers that specialize in research and often recruit healthy patients for early drug trials and compensate them. There are some online sites that say you can volunteer for as many studies as you qualify for in any given year.  There are obvious concerns about whether subjects need a break between trials as well as deception involved in gaining access to the trial, adhering to the protocol, and skewing the trial results (2-4).  I also have the concern that it is easy to fake some psychiatric disorders to get into a paid trial. The trials I have been in as an investigator were all through academic centers where reimbursement was primarily to cover travel expenses.  The professional patient role was probably less in those circumstances but it is an issue that is only covered in the research literature on a sporadic basis.

With the number of variables listed in the diagram – it takes a lot of thought to figure out the impact on the trial and clinical care.  It is useful to keep in mind a few overall points. First, there is a lot more going on in the trial than an effect based only on the biological effects of a medication (placebo, nocebo).  Second, clinical care is not like a clinical trial.  If someone is very ill and they are not responding to a medication or having side effects – a change needs to occur very quickly.  That is typically the same day on an inpatient unit or a day or two later in the outpatient clinic.  In a clinical trial, the subject can drop out or discuss other options with the investigator but those options are very limited and slow.  Major side effects typically need to be reported to the Human Subjects Board monitoring the research protocol, the protocol sponsor, and in some cases the FDA. Third, clinical care by its very nature incorporates many of the nonspecific factors associated with a nondrug positive response usually termed a placebo response. These same nonspecific factors can occur in psychotherapy trials as well as complementary medicine trials. 

Another important aspect of clinical trials is the measurement issue.  Various rating scales are typically employed as representations of psychiatric diagnoses. There are standardized and valid scales to measure adverse drug effects and, in some cases, more specific symptoms like sleep.  Global ratings by the investigators are added as an additional measure of validity.  These are all crude measures and unless subjects are selected specifically for a phasic severe disorder – the ratings will fluctuate significantly over the course of the trial based on various factors. In addition to the psychological, environmental, and biological factors some authors in medicine and biology have started to use the Heisenberg effect to describe this uncertainty. Since these measurements are not of discrete entities and cannot be represented by wave functions – this term is used at the level of a crude analogy for uncertainty in biological measurement.

Clinical care is much different.  There has only been a pretense of measurement.  There has been some publication of measurement-based psychiatry as an offshoot of evidence-based medicine.  I don’t think it offers much of an improvement over clinical psychiatry.  In psychiatry like all of medicine we are taught to recognize and respond to patterns not rating scales. I was reminded just this week of a referral I got of a patient who was described as having a histrionic personality disorder based on DSM criteria and some additional psychometric testing. It was apparent to me with a few minutes that the patient was manic and the correct diagnosis was bipolar disorder.  Psychiatrists are taught by seeing real patients, recognizing their problems, and organizing a treatment plan around those patterns. Referring to rating lists based on DSM symptoms that are in turn based on patterns in real people leaves a lot of room for interpretation.

The treatment aspect is also a critical difference.  There are few people who would want to receive treatment the way it is set up in clinical trials. People with any degree of a problem would not get adequate treatment. Most clinical trials are set up to see if there is an adequate efficacy and safety signal for the FDA to allow the medication on the market.  Post marketing surveillance is used to detect rare but severe side effects that might result in a new medication being withdrawn from the market.   

Clinical treatment is timely and effective in psychiatry.  Like the rest of medicine, it is not perfect and a lot can go wrong.  Based on my  experience as a researcher, a quality assurance reviewer, a teacher, and a colleague in large departments most people can count on finding a good if not excellent psychiatrist to provide effective treatment.  

 

 

George Dawson, MD, DFAPA

 

References:

1:  Piantadosi S.  Clinical Trials: A Methodological Perspective. 3^rd ed.  Hoboken, NJ: John Wiley and Sons, Inc, 2017: 886 pp.

2:  Shiovitz TM, Zarrow ME, Shiovitz AM, Bystritsky AM. Failure rate and "professional subjects" in clinical trials of major depressive disorder. J Clin Psychiatry. 2011 Sep;72(9):1284; author reply 1284-5. doi: 10.4088/JCP.11lr07229. PMID: 21951988.

3:  Pavletic A, Pao M. Safety, Science, or Both? Deceptive Healthy Volunteers: Psychiatric Conditions Uncovered by Objective Methods of Screening. Psychosomatics. 2017 Nov-Dec;58(6):657-663. doi: 10.1016/j.psym.2017.05.001. Epub 2017 May 9. PMID: 28651795; PMCID: PMC5680154.

4:  Lee CP, Holmes T, Neri E, Kushida CA. Deception in clinical trials and its impact on recruitment and adherence of study participants. Contemp Clin Trials. 2018 Sep;72:146-157. doi: 10.1016/j.cct.2018.08.002. Epub 2018 Aug 21. PMID: 30138717; PMCID: PMC6203693.

5:  Atkins PW.  Physical Chemistry. 5^th ed.  Oxford: Oxford University Press. 1994:  380-387.

 

Workforce Reference:  40,000 psychiatrists/290,000 nurse practitioners + 209,000 primary care physicians + 115,000 physician assistants – certified = 40,000/654,000 = 0.06 or 6%. 

Graphics:  I produced the above graphic with Visio.  Click to enlarge.