Recent Opinion About Diagnostic Heterogeneity – Gets It Wrong

There was an opinion piece about categorical diagnosis in psychiatry and diagnostic heterogeneity that was published in Psychiatric Research weeks ago (1), that generated a lot of controversy.  The controversy started when an online publication characterized the article as showing that Psychiatric Diagnoses Found to Be "Scientifically Meaningless".  The author of that article subsequently posted that the article was written by science undergraduates re-purposed as science writers.  If this was supposed to be investigative journalism it failed at several levels not the least of which is the apparent conflict of interest by the authors. Instead the internet article basically quotes the authors as factual and scientific rather than a rhetorical opinion piece.  What follows is my take on the Psychiatric Research Article.

The first sign of bias that a reader may encounter in the original article is right in the abstract. The concluding sentence reads:

“A pragmatic approach to psychiatric assessment, allowing for recognition of individual experience, may therefore be a more effective way of understanding distress than maintaining commitment to a disingenuous categorical system.” (my emphasis added).

When I read this sentence, it was difficult for me to believe that peer reviewers for a psychiatric journal could allow it to pass. In one sentence the authors are allowed to distort and discredit psychiatric clinical methods and diagnostic methods that have been carefully developed for over a century.  I won’t belabor the definition of “disingenuous” but it is safe to say that the expenditures in terms of brainpower and money as well as the transparency of the process make the production of the DSM 5 one of the more rigorous approaches to a diagnostic system in medicine. The people sitting on the DSM 5 committees for each section were acknowledged experts in their fields with decades of experience and published research.  Production of the DSM-5 was also a multiyear process that took 14 years to develop prior to its publication in 2015 (2).  During that time there was a multiyear grant that sponsored 13 international conferences on specific diagnostic issues.  Guiding principles and conceptual issues were examined.  Public input was solicited. Hundreds of clinicians and researchers were involved.  There was transparency about potential conflicts of interest. It was not just an intense effort – it was a unique diagnostic effort in terms of overall vigor and resource utilization.   Describing the output of all of this work as “disingenuous” and getting that in print lead me to question the peer review and editorial process.  Are the editors and reviewers ignorant of the effort that went into the diagnostic categories or don’t they care? It is clear that the authors of this article don’t.

The second red flag in the paper to anyone familiar with typical antipsychiatry arguments is the mention of Foucault and the suggestion that psychiatric classification occurs within wider sociocultural developments and that these roots have resulted in diagnostic heterogeneity.  In fact, Foucault’s observations of psychiatry were inaccurate at the time and have not held up at all over the course of time. The authors seem to ignore the actual reasons for categorical diagnosis in the first place and list none of those references.  Practically all modern DSM work can be traced back to the reference generally referred to as the Feighner criteria (3).  Reading those papers, clearly describes categorical diagnosis as a work in progress and the importance of diagnosis. The authors also describe five phases for the validation of psychiatric diagnoses.  They have this comment on diagnostic heterogeneity:

“In the absence of known etiology or pathogenesis, which is true of the more common psychiatric disorders, marked differences in outcome, such as between complete recovery and chronic illness, suggests that the group is not homogeneous. This latter point is not as compelling in suggesting diagnostic heterogeneity as is the finding of a change in diagnosis. The same illness may have variable prognosis, but until we know more about the fundamental nature of common psychiatric illnesses, marked differences in outcome should be regarded as a challenge to the validity of the original diagnosis.” p 57.

These authors suggested 5 phases to establish the diagnostic validity of psychiatric illness including the clinical description, laboratory studies, delimitation from other disorders, follow-up studies, and family studies.  There are entire texts dedicated to some of these markers on epidemiology and family studies.  One of the mandates of the DSM-5 committees was to review all of this data and compile it into the most clinically useful form.  In the interim they happened to pare the total number of diagnoses from a maximum of 297 in DSM-IV to 157 in DSM-5 (see reference 2, p xxiii).  This is the basis of categorical diagnosis – not the narrative of a philosopher.

Contrary to the idea that the current authors and the like-minded authors they have referenced have discovered diagnostic heterogeneity it has been widely acknowledged from the outset and by all current psychiatrists. There are no surprises here especially for people trained as physicians. Practically every complex biological illness is heterogeneous with heterogeneous outcomes as well as polygenic etiologies.  Their Foucauldian criticism also ignores the fact that the Washington University group was based on empirical research as opposed to the psychoanalytic process of the day.
 

The example of the empirical approach is illustrated by tracing the development of Major Depression criteria from 1950 to 1980. In fact, many in that group were highly skeptical of psychoanalysis as a possible diagnostic process at all. As they started to publish research article, one of their original articles was highly edited by a psychoanalyst/editor to remove any reference to the term diagnosis. 

The second acknowledged aspect of psychiatric diagnosis and treatment that is given short shrift by the authors is the issue the value of both diagnosis and formulation or as Kendler, et al discuss:

“However, neither we nor, we think, the developers of the criteria would claim that assessing operationalized diagnostic criteria is all there is to a good psychiatric evaluation. While critical, a diagnosis does not reflect everything we want to know about a patient. Our diagnostic criteria, however detailed, never contain all the important features of psychiatric illness that we should care about.” (see reference 4 p. 141.)

The authors’ research method is an exercise in subjectivity.  They basically read five chapters in the DSM 5 (schizophrenia spectrum and other psychotic disorders, anxiety disorders, bipolar and related disorders, trauma and stressor related disorders, and anxiety disorders) and use a technique called “thematic analysis” “to code themes or patterns of meaning across diagnostic categories being analyzed, with a particular focus on the heterogeneity or differences across types of diagnostic criteria.”  You don’t need an advanced research seminar to figure out what is wrong with that picture. Here is a group of psychologists several of whom make a career out of criticizing psychiatry and who are building a case that psychiatric diagnoses are inferior to their own vague diagnostic system using a qualitative technique that even their reference (5) refers to as having “no particular kudos as an analytic method – this, we argue, stems from the very fact that it is poorly demarcated and claimed, yet widely used”.  What outcome would any objective observer expect?

The combinatorics argument:

The authors make it seem like large combinations of diagnostic features mean categorical diagnoses are problematic.  Although they don’t say it explicitly - referring to more diagnoses greater than the number of stars in the solar system - suggests improbability.  But do large combinations of number preclude reasonable human use?  A chess board for example has an 8 x 8 square configuration and by some estimates - 10¹³⁷ moves are possible.  And yet players at all levels seem to be able to negotiate a chess board and determine win, lose or draw.  Master players can develop strategies that make them more likely to win.  Is there similar evidence that diagnoses with large combinations can be managed the same way?  What follows is a mixed table of a psychiatric diagnosis (PTSD) that yields a large number of combinations of diagnostic criteria on the left, a dimensional scale for depression (DEP) from a standard psychological test (MMPI), two different criteria for systemic lupus erythematosus (SLE), and criteria for asthma. Qualifiers for each column are listed at the bottom.

Disorder	PTSD (1)	MMPI-DEP (2)	SLE (ACR) (3)	SLE (SLICC) (4)	Asthma (5)
Criteria	Presence of 1 (or more) of the following symptoms: 1. 2. 3. 4. 5. One or both of the following symptoms: 1. 2. Two (or more) of the following: 1. 2. 3. 4. 5. 6. 7. Two (or more) of the following: 1. 2. 3. 4. 5. 6.	15/26 items	4 of 11 criteria: 1. 2. 3. 4. 5. 6. 7.   A or B 8.   A or B 9.   A or B 10. A or B or C or D 11. A or B or C or D or E	4 of 17 criteria including at least 1 clinical criterion and 1 immunologic criterion; or biopsy proven lupus nephritis: 1.   A or B 2.   A or B 3.   4.   A or B 5.   A or B 6.   A or B 7.   A or B 8.   A or B 9.   10. A or B 11. 12. 13. 14. 15. 16. A or B 17.	1. A or B or C or D 2. A1 or A2 or A3  or B or C
Minimal Combinations	3,150	7.726160e6	330	2,380	36
Total Possible Combinations	636,120	7.726160e6 + 5.311735e6 + 3.124550e6 + 1.562275e6 + 657800 + 230230 + 65780 + 14950 + 2600 + 325	12,555	321,489	46

Footnotes:

1.  This column is from the reference: Galatzer-Levy, I.R., Bryant, R.A., 2013. 636,120 Ways to have posttraumatic stress disorder. Perspect. Psychol. Sci. 8, 651–662.

2.  I have several opinions from different psychologists on the current use of this MMPI scale and the raw cut-off scores. I understand that there are different raw scores for men and women. I can recalculate this scale based on any numbers that may be deemed more reliable. Just email them to me along with the evidence.

3.  American College of Rheumatology (ACR) classification criteria for Systemic Lupus Erythematosus

4.  Systemic Lupus International Collaborating Clinics (SLICC) proposed revised classification criteria for Systemic Lupus Erythematosus

5.  There are numerous endophenotyping classifications for asthma.  It is clear at this point there is no comprehensive system of clinical classification.

What can be observed from this table?   

Apart from waxing poetically they seem to not recognize that common psychological approaches scale to an even larger extent – much greater than 10¹⁸. I have also demonstrated that the way diagnostic criteria are worded makes a big difference in counting word combinations.  Just using the DSM phrasing “or more” greatly increases the number of combinations.  Criteria designed like the SLE criteria as a series of “A or B” choices that greatly reduce the number of possible combinations.  On the other hand dimensional criteria like a single scale from a popular psychological test – greatly increases the number of possible combinations because that scale is a many n and many k.  Using a 15/26 item scale results in 10⁷ combinations.  Using that as a ball park estimate for the other clinical scales results in numbers far larger than used by the authors to criticize categorical diagnosis.  The other aspect of this table is that less combinations is not necessarily better. With asthma for example, these numbers are based on very basic diagnostic criteria.  There are at least 2 other 6 item endophenotype systems and an additional cough variant asthma, but currently experts in the field have not developed a way to incorporate that level of clinical complexity into diagnostic criteria that would be useful to clinicians. Low number of combinations of diagnoses criteria are not necessarily better than higher numbers – especially when the disease complexity is not captured.  

The second issue with combinatorics is that they are not predictive of anything. Great strides in treating post-traumatic stress disorder have occurred in the past 30 years using criteria with a high number of combinations.  That obviously does not preclude patient selection or monitoring in clinical trials of either psychotherapy or pharmacotherapy. It does not prevent the successful diagnosis and treatment of patients in clinical settings in many cases where severe and potentially fatal psychiatric illness exists.  As an example, delirious mania had a fatality rate of 75% in 1849 in the United States (7). That number has fallen to zero with psychiatric treatment based on categorial diagnosis and the clinical training of psychiatrists to recognize severe illness. Many of those improvements have occurred in the past 30-50 years. 

  

In the authors selection strategy, large sections of the DSM 5 that clearly disprove the author’s contentions are omitted. The elimination of Neurocognitive Disorders, Sleep-Wake Disorders, and Substance Related and Addictive Disorders for example also eliminates biological markers and autopsy validation of criteria of diagnoses.  Table 1 (p. 482 of DSM-5) contains 127 discrete categorical diagnoses across 10 categories of substances. 

But the larger misunderstanding here is that what the authors disparage as heterogeneity is an expected part of medicine. Every physician knows that no two patients with asthma, benign prostatic hypertrophy, or gout are the same. There are a collection of illness features with some overlap but no truly homogeneous categories – even in clinical trials that attempt to minimize it. Biological systems especially the brain are designed to scale in various ways including based on combinatorics of various biological elements.  The author’s use of the term quadrillion, happens to be the estimated number of synapses in the brain but that is just a starting point of how systems in the human brain can scale.  The endothelial system in the human body has more cells than the brain and massive heterogeneity that allows for regulation of the vascular beds the human body. The hematopoietic and immune systems have similar levels of scaling that could also result in very large number of combinations. In none of these cases do the number of combinations of cell types, connections, tissue behavior, or descriptions preclude diagnoses, research or treatment.  A very small sample of this heterogeneity is suggested by the table below.  

Heterogeneity In Normal Functioning And Disease States In Human Biology (very partial list)

Endothelial cells	Diabetic nephropathy
Hematopoietic Stem Cells	Hepatitis C virus
Neuroendocrine Neoplasms	Ischemic Stroke
Leukemia - Clonal and Intraclonal cell types	Prostate Cancer
Aphasia syndromes	Mitochondrial Myopathies
Atrial Fibrillation Syndromes	Asthma
Immunodeficiency syndromes	Coriticobasal Degeneration
Diabetes Mellitus Type I and II	Viral Syndromes
Congestive Heart Failure	Cryptospridium genus and species

The authors ignore clinical heterogeneity that physicians have to address in their patients every day.  Very few physicians see clinical trials subjects as patients requesting assistance. That means comorbid physical illnesses, variations in patient tolerance of medical and psychological interventions, pharmacokinetic and pharmacodynamic factors, heart disease, liver disease, renal disease, substance use disorders, traumatic brain injuries, old age, pediatric age, suicide risk, aggression risk, impaired functional capacity, and even pregnancy have to be addressed in patients being seen every day by psychiatrists and adjustments have to be made. Only physicians schooled in heterogeneity would be able to treat those people.  Only physicians schooled in heterogeneity would realize that the people in clinical trials are rarely the people being seen in the office.  

In conclusion, the authors have a poor understanding of diagnostic heterogeneity and why it is a central part of medicine.  Some of their arguments are similar to arguments offered up by the critics of Kraepelin in the early 20^th century.  Other arguments - like the combinatorial ones reflect a poor understanding of biological systems and how they scale as well as a lack of understanding of medicine. Physicians know for example that diagnostic models are not completely explanatory, that over time - the explanations change, but that science exists at some level of that explanation or treatment. That is the nature of biological as opposed to physical systems. Anyone interested in these issues can find a rich literature out there that describes these problems and even the involved philosophy. Unfortunately, only one of the authors referenced (out of 28) is written by anyone authoritative in that area.

The only disappointment greater than an article like this being published is the fact that it was published in the journal Psychiatric Research.  It has little to do with psychiatry or research and it is shocking that the obvious problems with article were overlooked. On the other hand, this journal was never at the top of my reading list and this may be why.

George Dawson, MD, DFAPA

References:

1: Allsopp K, Read J, Corcoran R, Kinderman P. Heterogeneity in psychiatric diagnostic classification. Psychiatry Res. 2019 Sep;279:15-22. doi: 10.1016/j.psychres.2019.07.005. Epub 2019 Jul 2. PubMed PMID: 31279246.

2:  Black DW, Grant JE.  DSM-5 Guidebook. American Psychiatric Publishing, Arlington, VA: pp 543.

3: Feighner JP, Robins E, Guze SB, Woodruff RA Jr, Winokur G, Munoz R. Diagnostic criteria for use in psychiatric research. Arch Gen Psychiatry. 1972 Jan;26(1):57-63. PubMed PMID: 5009428.

4: Kendler KS, Muñoz RA, Murphy G. The development of the Feighner criteria: a historical perspective. Am J Psychiatry. 2010 Feb;167(2):134-42. doi: 10.1176/appi.ajp.2009.09081155. Epub 2009 Dec 15. PubMed PMID: 20008944.

5: Braun, V., Clarke, V., 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101. https://doi.org/10.1191/1478088706qp063oa.

6: Kendler KS, Engstrom EJ. Criticisms of Kraepelin's Psychiatric Nosology: 1896-1927. Am J Psychiatry. 2018 Apr 1;175(4):316-326. doi: 10.1176/appi.ajp.2017.17070730. Epub 2017 Dec 15. PubMed PMID: 29241358.

7: Bell, L., 1849. On a form of disease resembling some advanced stage of mania and fever. Am. J. Insanity 6, 97–127.