More Lessons From Dermatology......

 

I have several dermatological ailments and fortunately a very good clinic of dermatologists.  The main problems have been atopic dermatitis (eczema) that did not start until I was 65 and rosacea starting slightly later.  I have posts on the pathophysiology of these conditions at the links.  I have the predisposing factors including genetics, ethnicity (fair-skinned individuals of Celtic and northern European heritage (Fitzpatrick skin types I–II), and associated conditions (childhood and adult allergic asthma) but did not get the associated atopic dermatitis at the time.  One of my siblings had both in childhood.  My father probably had rosacea but he lived during a time when dermatology treatment was generally not done by dermatologists and quite primitive. By primitive I mean, physicians tried to excise inflammatory areas from his face instead of treating them medically like they do now.

The global prevalence of rosacea is estimated at 5.46% (4).  Women are affected more than men and peak prevalence occurs in middle age (45-60 years old).  In a study that looked at how many people were untreated – the prevalence was 12.3% in Germany and 5% in Russia.  Nearly half of those affected had not received any care in the previous year despite 1/3 endorsing a significant effect on quality of life (5).  In addition to cosmetic effects rosacea is a cause of dry eyes and other significant ocular complications and sensitive skin lowering the threshold for pain and irritation.

I was doing quite well until I needed a change in sleep apnea treatment.  Three months ago I changed from APAP to BiPAP because of an increasing AHI.  Because it was running at much higher pressure, they suggested using a mask rather than nasal CPAP.  That mask led to a flareup of rosacea that required a month of treatment with doxycycline.  But even after that treatment I was left with a 4-5 mm inflammatory nodule beneath my right eye that did not resolve. 

I saw a new dermatologist today and the conversation went something like this:

Me:  “This started about 3 months ago when I tried a CPAP mask and it caused a flare-up of rosacea.  This nodular area did not clear with doxycycline.  At about 2 weeks the area of inflammation extended up into the orbital area but stopped there ever since.

Derm:  “Are you still using the triple cream?”

Me:  “Yes twice a day.  It generally works great.”

Derm:  Inspects the area with a dermatoscope in detail and then: ’Yes, it looks like inflammation.  There may be some microabscesses in the area.  It does not look like cancer or infection.  What we need to do is try a different antibiotic for a month and then if it doesn’t clear up – do a biopsy.  I am going to prescribe cefuroxime after we make sure there are no drug interactions.  I notice you are on flecainide.  We were told never to prescribe it so I want to make sure it does not interact.”

His scribe ran a drug interaction check. I set up an appointment to see him in a month and picked up the prescription for cefuroxime.  On the way to the pharmacy, I recalled enrolling patients in a cefuroxime trial for urinary tract infections.  And then I tried to recall all of the serious side effects of cephalosporins – the class of antibiotics that cefuroxime is in.  That is just the way my mind works. 

What are the lessons about psychiatry here?  I don’t think the lessons are for psychiatrists because we know better.  The lessons are basically to counter all of the misinformation about psychiatric treatment and medications from antipsychiatrists, health and wellness influencers, and other critics who don’t seem to know very much about the field.  Here goes:

1:  Diagnoses are not easy – and experts are more likely to make them and even then most are provisional.  In all my teaching about diagnostic thinking in medicine pattern matching is a significant component.  Dermatologists and ophthalmologists are the examples I typically used comparing their diagnoses to other physicians. In this case, the question is what any other physician would have diagnosed the mark on my face as and how it would have been treated.  I have actually been there and done that and it would vary from no diagnosis or treatment to acne and metronidazole. 

2:  Transdiagnostic – yes probably – you would think the term had been invented for psychiatry in the last 10 years.  It is typically used as a criticism of categorical diagnosis as in “there are just so many transdiagnostic symptoms nothing is specific?”  And "all of this comorbidity is a strike against categorical diagnoses."  The reality is there are many so-called transdiagnostic symptoms across all of medicine and many of them are more robust than psychiatric symptoms.  Rash is one of the more robust.  Rashes are transdiagnostic across the 2,000 to 3,000 conditions that produce similar rashes across disorders as well as many different rashes within the same category in the same person.  There are many different rashes (intermediate phenotypes) presenting as rosacea for example, in this case a solitary inflammatory papule.

3:  No labs?  There is no lab test for rosacea or most dermatology conditions. They are clinical diagnoses made on that basis considering all of the findings at the time of the exam.

4:  Is there a biopsy result specific for rosacea?  This is a familiar criticism of psychiatric diagnoses – there is no specific test that rules in the diagnosis. From reference 1 below: “A skin-biopsy specimen is obtained only to rule out other diagnoses, since the histopathological features of rosacea are typically not specific to rosacea.”  Rosacea is a clinical diagnosis based on history and phenotypic criteria and no specific diagnostic test is needed to confirm it (3).

4:  Prevalence and Quality of Life Considerations – prevalence and quality of life considerations for rosacea and common psychiatric disorders are similar. 

 

Rosacea studies looked at pooled prevalence (5.46%) and geographic prevalence (Germany 2.1–12.3%, Russia 5.0%, U.K. incidence rate 1.65 per 1,000 person-years (the only study to quantify incidence) rather than interval prevalence (4-6).  Rosacea is not reported in the same intervals as psychiatric disorders because it is considered a chronic relapsing condition even though a segment of these specific psychiatric disorders has that same property.  There is no cure - another frequent criticism of psychiatric diagnoses.  

Although dermatology and psychiatry use different quality of life (QoL) impairment scales, in the respective disorders about 11% of rosacea patients report severe impairment and anxiety and depressive disorders report ranges of 26-85% using a cut-off of 2 standard deviations over average community ratings (7).  There are also comorbidity considerations with high percentages of rosacea patients reporting significant levels of depression and anxiety. 

5:  Under and missed diagnoses – deference to expert diagnosis is a time-honored tradition in medicine with a more recently established empirical basis. Overall diagnostic accuracy for dermatology conditions is 37-57%.  Roughly ½ of these conditions are incorrectly diagnosed in primary care compared with dermatologists (8). That rate of misdiagnoses is similar to the rate for anxiety and depressive disorders in primary care of 40-50% (9-11).

6:  Under treatment – undertreatment follows underdiagnosis in most cases, but undertreatment can also occur when the diagnosis has been established.  In the case of rosacea delayed diagnosis can lead to progressive (granulomatous) disease and ocular complications. Seborrheic dermatitis is also a frequently co-occurring condition and patients are often unaware that they have this condition as well.

Not treating depression and anxiety on a timely basis leads to similar chronicity and conditions more resistant to treatment.  It increases both the risk of suicide and self harm with chronicity. Untreated depression and anxiety are risk factors for cardiovascular disease and substance use. Chronic pain can be increased.  Both conditions are well documented causes of significant disability.

7:  Uncertain pathophysiology – The pathophysiology of most psychiatric disorders where the possible cause has been ruled out is not known.  The same is true for rosacea.  In any similar group of medical disorders there are commonly suggested hypotheses that can be grouped by general mechanism as indicated in the table below:

8:  Uncertain medication mechanism of action/placebo response -  Since the underlying pathophysiological is unknown the mechanisms of action of the recommended treatments is unknown for rosacea and psychiatric disorders.   This means that clinical trials are needed to test the efficacy and safety of treatments and clinical care follows.

Comparisons of response rates in a selection of antidepressant and rosacea medication trials show significant placebo response in both with a slightly higher response rate in the rosacea trials (66% v. 50%).  At the same time the metrics used for effect size in both tables are not comparable.  If we change to a comparable metric like Number Needed to Treat (NNT) we see ranges of 3-8 for rosacea and 4-6 for antidepressants.  On that basis it is fair to say that response rates to rosacea medication and antidepressants are generally comparable.

The strict comparison is limited by the fact that studies have different outcome measures.  Rosacea studies use a clinician rated global improvement score.  Antidepressant trials may also have a global improvement score but more likely use clinical scales like the HAM-D or MADRS.  Both types of ratings have a consensus marker for improvement but they are not calibrated against one another.    The placebo response rates may have different mechanisms.  Both may have a regression to the mean and clinical care/therapeutic alliance component but the rosacea trials can also be affected by atmospheric conditions and additional topicals that can affect skin moisture.  Rosacea trials tend to be longer than antidepressant trials (12-16 weeks versus 8-12 weeks).

There is a question of real-world effectiveness with both conditions.  It has been studied in depression (12). It was found that for MDD, there is a 90% gap between those with the diagnosis and this receiving effective treatment (41.8% receive treatment and of those only 23.2% of those diagnosed received effective treatment.)  The RISE study (5) suggests that in a screened population for rosacea,  80% were never previously diagnosed.  Of the 20% who were 47.5% had received no rosacea care whatsoever, and only 23.7% had received topical and/or systemic drugs suggesting similar underdiagnosis and treatment as depression.   

Whenever clinical trials of antidepressant are discussed, some critics say that the lack of hard outcomes (all-cause mortality, cause-specific mortality (MI, stroke, suicide), hospitalization) as opposed to symptom-based outcomes is a major problem.  In the comparison with rosacea – all of the outcome measures are symptom-based and no evidence that treatment prevents associated or long-term complications like rhinophyma, telangiectasia, or ocular complications.  There are register based/naturalistic studies (outside of clinical trial design) that show long term use of antidepressants reduces all cause mortality, suicidal ideation, and cardiovascular mortality (13-17).  Not all analyses agree and in some cases the argument was made that it was an antidepressant class effect (18,19).  Rosacea on the other hand has not been studied against an all cause mortality endpoint because it is not associated with increased mortality.    

9:  On label - off-label -  There are currently 40 FDA approved medication for depression and 10 FDA approved medications for rosacea.  Antidepressant development dates back to the 1950s  when it was discovered that medications used to treat tuberculosis also had positive effects on mood.  Rosacea medication development began as an early topical antibiotic treatment (Sodium sulfacetamide 10%/sulfur 5% for papules/pustules).  That early treatment was before 1962.  The next development did not occur until topical metronidazole in 1988.  The only oral antibiotic approved is doxycycline 40 mg MR (Oracea) that is a combination of doxycycline 30 mg IR and doxycycline 10 mg delayed release (DR).  There are no combination medications approved despite the fact that there are compounded formulations.  As an example the Rosacea triple cream contains metronidazole, ivermectin, and azelaic acid.  Each component is FDA approved for monotherapy.    

10: Politicalization – the only real criticism I have seen of dermatologists was comedic. In an episode of Seinfeld, Jerry was trivializing what dermatologists do until he was reminded that they diagnose and treat skin cancer.  Despite the parallels to psychiatry, they have no anti-factions or health and wellness influencers suggesting they are creating more problems than they solve or negative media coverage or high visibility criticism by experts in their own field.  The head of HHS is not suggesting their treatments are overprescribed.

There is no great agitation over dermatology – their methods or treatments despite similar levels of uncertainty and clinical methods with psychiatry.  I am not suggesting there should be.  I am suggesting that psychiatry should be approached by outsiders with the same levels of acceptance that they have for dermatology.  I am also suggesting that if you have a skin condition and nobody seems to be able to diagnose or treat it – see a dermatologist. The advice applies to a mental disorder.  See an expert in that field. 

I know this does happen but there is always a delay.  And there is always plenty of misinformation about the field.  As I have posted here before – practically all of the people I saw over my 40-year career had seen somebody else first – often many different people over a number of years before they decided to see a psychiatrist. It was often the result of a referral decision.  But most importantly – don’t believe what you read in the papers whether it is health and wellness advice or recommendations by the current Secretary of HHS. 

If there is a serious problem with your mental state – see the right person.

 

George Dawson, MD, DFAPA

 

1:  van Zuuren EJ. Rosacea. N Engl J Med. 2017 Nov 2;377(18):1754-1764. doi: 10.1056/NEJMcp1506630. PMID: 29091565.

2:  Frazier W, Zemtsov RK, Ge Y. Rosacea: Common Questions and Answers. Am Fam Physician. 2024 Jun;109(6):533-542. PMID: 38905551.

3:  Dirr MA, Ahmed A, Schlessinger DI, et al. Rosacea Core Domain Set for Clinical Trials and Practice: A Consensus Statement. JAMA Dermatol. 2024 Jun 1;160(6):658-666. doi: 10.1001/jamadermatol.2024.0636. PMID: 38656294.

4:  van Zuuren EJ, Arents BWM, van der Linden MMD, Vermeulen S, Fedorowicz Z, Tan J. Rosacea: New Concepts in Classification and Treatment. Am J Clin Dermatol. 2021 Jul;22(4):457-465. doi: 10.1007/s40257-021-00595-7. Epub 2021 Mar 23. PMID: 33759078; PMCID: PMC8200341.

4:  Gether L, Overgaard LK, Egeberg A, Thyssen JP. Incidence and prevalence of rosacea: a systematic review and meta-analysis. Br J Dermatol. 2018 Aug;179(2):282-289. doi: 10.1111/bjd.16481. Epub 2018 May 31. PMID: 29478264.

5:  Tan J, Schöfer H, Araviiskaia E, Audibert F, Kerrouche N, Berg M; RISE study group. Prevalence of rosacea in the general population of Germany and Russia - The RISE study. J Eur Acad Dermatol Venereol. 2016 Mar;30(3):428-34. doi: 10.1111/jdv.13556. PMID: 26915718; PMCID: PMC5067643.

6:  Hilbring C, Augustin M, Kirsten N, Mohr N. Epidemiology of rosacea in a population-based study of 161,269 German employees. Int J Dermatol. 2022 May;61(5):570-576. doi: 10.1111/ijd.15989. Epub 2021 Dec 12. PMID: 34897653.

7:  Rapaport MH, Clary C, Fayyad R, Endicott J. Quality-of-life impairment in depressive and anxiety disorders. Am J Psychiatry. 2005 Jun;162(6):1171-8. doi: 10.1176/appi.ajp.162.6.1171. PMID: 15930066.

8:  Bridges C, Morris C, McElroy JA, Quinn K, Dyer J, Becevic M. Utility of Dermatology Extension for Community Healthcare Outcomes (ECHO) sessions in the adult and paediatric population. J Telemed Telecare. 2021 Jul;27(6):376-381. doi: 10.1177/1357633X19874200. Epub 2019 Sep 16. PMID: 31526083.

9:  US Preventive Services Task Force. Screening for Depression and Suicide Risk in Adults: US Preventive Services Task Force Recommendation Statement. JAMA. 2023;329(23):2057–2067. doi:10.1001/jama.2023.9297

10:  Jackson-Triche  ME, Unützer  J, Wells  KB.  Achieving mental health equity: collaborative care.   Psychiatr Clin North Am. 2020;43(3):501-510. doi:10.1016/j.psc.2020.05.008

11:  Wang  PS, Angermeyer  M, Borges  G,  et al.  Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization’s World Mental Health Survey Initiative.   World Psychiatry. 2007;6(3):177-185.

12:  Vigo D, Haro JM, Hwang I, et al. Toward measuring effective treatment coverage: critical bottlenecks in quality- and user-adjusted coverage for major depressive disorder. Psychol Med. 2022 Jul;52(10):1948-1958. doi: 10.1017/S0033291720003797. Epub 2020 Oct 20. PMID: 33077023; PMCID: PMC9341444.

13:  Chan JKN, Solmi M, Lo HKY, Chan MWY, Choo LLT, Lai ETH, Wong CSM, Correll CU, Chang WC. All-cause and cause-specific mortality in people with depression: a large-scale systematic review and meta-analysis of relative risk and aggravating or attenuating factors, including antidepressant treatment. World Psychiatry. 2025 Oct;24(3):404-421. doi: 10.1002/wps.21354. PMID: 40948054; PMCID: PMC12434377.

14:  Lagerberg T, Fazel S, Sjölander A, Hellner C, Lichtenstein P, Chang Z. Selective serotonin reuptake inhibitors and suicidal behaviour: a population-based cohort study. Neuropsychopharmacology. 2022 Mar;47(4):817-823. doi: 10.1038/s41386-021-01179-z. Epub 2021 Sep 24. PMID: 34561608; PMCID: PMC8882171.

15:  Gusmão R, Quintão S, McDaid D, Arensman E, Van Audenhove C, Coffey C, Värnik A, Värnik P, Coyne J, Hegerl U. Antidepressant Utilization and Suicide in Europe: An Ecological Multi-National Study. PLoS One. 2013 Jun 19;8(6):e66455. doi: 10.1371/journal.pone.0066455. PMID: 23840475; PMCID: PMC3686718.

16:  Korkeila J, Salminen JK, Hiekkanen H, Salokangas RK. Use of antidepressants and suicide rate in Finland: an ecological study. J Clin Psychiatry. 2007 Apr;68(4):505-11. doi: 10.4088/jcp.v68n0403. PMID: 17474804.

17:  Pan YJ, Yeh LL. Associations between mortality and exposure to psychotropic medication: A population-based cohort study for depressive disorders. Aust N Z J Psychiatry. 2023 Sep;57(9):1253-1262. doi: 10.1177/00048674221145337. Epub 2023 Jan 11. PMID: 36629047.

18:  Zhou S, Wang C, Zhang Y. Antidepressant use and all-cause mortality in depressed individuals: A real-world cohort study. PLoS One. 2025 Jul 11;20(7):e0327844. doi: 10.1371/journal.pone.0327844. PMID: 40644427; PMCID: PMC12250549.

19:  Zhuang X, Chen W, Zhan Y, Feng X, Liu C. Antidepressant selection modifies survival in depression: A National Cohort Study Using NHANES 2005 - 2018 data. Gen Hosp Psychiatry. 2026 Jan-Feb;98:33-40. doi: 10.1016/j.genhosppsych.2025.12.001. Epub 2025 Dec 3. PMID: 41351935.

20:  Andrade C. A Primer on How to Critically Read an Observational Study on Adverse Medical Outcomes Associated With Long-Term Antidepressant Drug Use. J Clin Psychiatry. 2022 Dec 7;83(6):22f14733. doi: 10.4088/JCP.22f14733. PMID: 36479952.

21:  Egeberg A, Fowler JF Jr, Gislason GH, Thyssen JP. Nationwide Assessment of Cause-Specific Mortality in Patients with Rosacea: A Cohort Study in Denmark. Am J Clin Dermatol. 2016 Dec;17(6):673-679. doi: 10.1007/s40257-016-0217-1. PMID: 27480418.

22:  Egeberg A, Hansen PR, Gislason GH, Thyssen JP. Assessment of the risk of cardiovascular disease in patients with rosacea. J Am Acad Dermatol. 2016 Aug;75(2):336-9. doi: 10.1016/j.jaad.2016.02.1158. PMID: 27444070.

The Semantic Memory of Physicians - and More...

 

I have the somewhat grandiose plan to model psychiatric diagnosis based on the cognition of a physician rather than focusing on the externals.  By the externals I mean classification systems and critiques of classification systems.  At a later date – I might try to comment on how this approach compares with AI.  For now, I will try to keep it focused on human diagnosticians.  I have an interest in this is because I have made and witnessed incredible diagnoses and treatments by physicians and psychiatrists who I have been affiliated with. I don’t think there has been much of a focus on the process.  A secondary consideration is that cognitive neuroscience is a neglected subject in psychiatry and I hope to make the point that should change. I would go as far as suggesting that cognitive neuroscience should be taught to all psychiatrists more urgently than focusing on another DSM.    

Since the early 1970s, memory functions are divided along various lines clinically and functionally. The first division is long term memory and working memory (also called short term memory).  On the long-term side there is a further division to declarative and procedural memory.  Declarative memory is divided into episodic and semantic memory.  Episodic memory is the ability to recall discrete events.  Semantic memory can have a number of graded definitions.  A minimalist definition is factual knowledge independent of the source (7). A definition more informed by recent research in cognitive psychology: “General (encyclopedic) knowledge as well as schematic representations of events distilled from lifelong experiences, retrieved independently from their original spatial or temporal context” (9).  The authors in that case give examples of knowing who wrote the book “1984” and what generally happens at a birthday party.  That naturally raises the question how does all of this freestanding knowledge occur in the first place?  And also – does that imply a connection to episodic memory? In other words, does semantic memory occur when the context surrounding episodic memory is forgotten?

In the case of physicians there is a very long list of formative experiences across the course of one’s career.  The ability to recall them often assists in making diagnoses and provides an advantage over a physician who has not experienced that event.  Semantic memory is about concepts, words, and their relationship independent of a specific event or experience.  It typically consists of a collection of general facts and word meanings.  For example, it would include facts that apples can be red, green, or yellow and what a mechanic does.

Anyone familiar with cognitive screening examinations has probably asked questions focused on semantic memory.  Naming, word similarities, verbal fluency by word generation, general knowledge questions, are all examples. 

The semantic memory of a physician will contain many unique concepts and they will vary based on experience and exposure to clinical scenarios.  The general categories can be described as the following:

1:  Meaningful prior experiences – even though episodic memory stores specific events at specific intervals, semantic memory contains the specific meaning.  In the case of psychiatry an example would be seeing the effects of CMV encephalitis in a major university transplant unit and a decade later seeing similar behavior and consulting on a case in a general community hospital for similar findings.  That similarity triggers non-analytic hypothesis generation.

2:  Prototypes - the patterns noted in the above example can be averaged over a group of patients and those averages can be consolidated into prototypes.  In the above case a psychiatrist may have seen many cases of encephalitis and many cases of meningitis resulting in encephalitis and meningitis prototypes.  Similar prototypes may exist for all major neurological, medical, and psychiatric condition that they have encountered.  Note that the prototype differs from diagnostic criteria (the typical focus) because it is recall of all of the relevant and in many cases unique clinical features that were experienced.

3:  Specific patient memories (exemplars) – all physicians recall specific patients.  These memories are important for non-analytical reasoning like pattern matching.

4:  Knowledge Encapsulation – medicine like most professions is based on a system of graduated learning.  Basic science transitions rapidly into clinical medicine and then into clinical practice and lifelong learning.  At each stage prior knowledge is reorganized in a more efficient way.  In this case – general biomedical knowledge from basic science is organized under higher level concepts. 

An example in one of the references is a person with an infection who is experiencing progressive physiological problems.  At the medical student/basic science level the analysis might proceed from the basic science level and pathophysiology first.  At the clinician level the relevant pathophysiology is organized as sepsis and that provides a more immediate pathway for intervention.  The encapsulation encompasses and efficiently organizes the lower-level information.  At the same time experts must retain a significant amount of that earlier information.  

5:  Illness Scripts – are mental representations of diseases containing three different dimensions.  The first is enabling conditions like risk factors, demographics, predisposition, and context.  The second is fault or underlying pathophysiology.  The third is consequences including signs, symptoms, lab findings, and course or natural history.  Experts have a significant collection of these features.   

One of the questions in this area is what kind of illness script do physicians have?  Should they all be from their particular specialty or should these scripts encompass the totality of their training?  Some authors suggest that the pathophysiological mechanisms from basic science needs to be retained for true expertise – so my conclusion is that the illness scripts from the entirety of a physicians training probably remain relevant.

This is important in psychiatry because the general pathophysiology important in today’s environment was probably not taught is any detail in medical school and most conditions that are not secondary to medical conditions or the effects of drugs do not easily lend themselves to physiological explanations.  I would suggest that medical stability, generalized seizures and seizure variants, increased intracranial pressure, meningitis, encephalitis, cerebral localization, cerebellar dysfunction, peripheral neuropathies, coma, confusion/stupor/delirium, intoxication, and cranial nerve deficits are some of the illness scripts that every psychiatrist must have.

6:  Semantic Qualifiers - every physician has a lexicon of semantic qualifiers acquired in both medical school and post graduate training. They include anatomic descriptions (areas, more specific locations), pathological descriptions, disease course descriptions, and many others. Framing clinical scenarios with these qualifiers is often all that is needed to acquire associations to the disease of interest.

7:   Base rates and Context – experts by way of their clinical practice have an intuitive grasp of the base rates of various clinical conditions and how they typically present in their practice.  These rates of presentations and findings are integrated with the other features of semantic memory (disease scripts, patterns, etc) for more analysis and hypothesis generation.

These features of semantic memory are of course models of brain function for the most part determined by experimental models in cognitive psychology. Examples include testing for specific functions and seeing how those modelled functions vary among trainees and experts at various stages of development. 

Apart from the descriptive approaches used in many studies on physicians at various levels of training are there any more general models that could apply?  Cognitive neuroscience and cognitive psychology offer a more complete model of memory and knowledge structures as well as the underlying biology.  The lead figure for this post is a case in point and has the potential to consolidate many of the descriptions under a more comprehensive model based on experimental validation.

At levels B and C in the diagram we see a perceptual episode being processed from the left to the right in the diagram.  The activated or instantiated schema is a template for extracting relevant features and repressing irrelevant features.  In the diagram circles represent general concepts and squares are action scripts. Gist in the case of the model is a representation of a single episode where much of the detailed information is removed.  The overall sequence at level B depicts how a schema serves to form semantic type memory (gists) and at the same time can be altered or accommodated by new information.

Level A in the diagram illustrates what is known about the localization of these processes largely from human fMRI and preclinical studies.  Memory schemas are stored in various sites including the retrosplenial cortex (RSPL), middle and superior temporal gyrus (MTG/STG), anterior temporal lobe (ATL), and temporoparietal junction (TPJ).  These sites are bound per the diagram to the ventromedial prefrontal cortex (vmPFC).    Solid lines are context sensitive associative pathways biased by the vmPFC. Broken lines in the diagram represent context irrelevant associations that are not activated or inhibited.

How might all of this model work for psychiatry?  In general physicians are seeing a lot of patients in their training and practice.  In the course of that work - schemas are developed for diagnoses, signs, symptoms, and situations.  Here is a comparison of two scenarios that all psychiatrists are trained to recognize acute encephalitis and bipolar disorder, manic with psychotic features. 

	Encephalitis	Bipolar disorder, manic with psychosis
Schema	Acute illness, acute altered mental status, fever, seizures, focal neurological deficits, CSF/MRI abnormalities	Acute illness, euphoria/irritability/anger, hyperactivity, functional impairment, psychosis, temporal pattern, exclusion features
Subschema	Predisposing factors, pathophysiology patterns, temporal pattern	Euphoric expansive Irritable dysphoric Spontaneous v. precipitated
Gist	Acute confusion + fever + temporal lobe MRI changes = treat as HSV until proven otherwise" "Young woman + new psychosis + movement disorder = think anti-NMDAR, look for teratoma" "Summer encephalitis + flaccid paralysis = arboviral, likely West Nile" “Immunocompromised man with acute agitation = think CMV encephalitis	Episodic psychosis +/- mood changes (diagnosis gist) Mood stabilizer + antipsychotic (treatment gist) Severe postpartum psychosis = think bipolar disorder, manic with psychotic features Catatonia – think bipolar disorder, manic/depressed/mixed with psychotic features.

 

I came up with the following graphic (click to enlarge) based on the descriptive categories and the cognitive neuroscience model of Gilboa and Marlatte (12).  From left to right – the  “heterogenous construct supported by clinical utility” characterization is probably the most charitable one from philosophers.  Others like “this disorder does not exist” or “this disorder is not real” are two additional examples.  The central semantic memory category includes investigations and models of diagnostic reasoning conducted largely on medical students and physicians.  The cognitive neuroscience model contains schema and I have attempted to show how the concepts and actions map from the semantic memory to the schema model.  In both the semantic memory and cognitive neuroscience model, although the focus is memory the conceptualizations are really knowledge structures emphasizing a dynamic role for the schema in incorporating features of reality – in this case patient encounters. The cognitive neuroscience and semantic memory models also map on to brain anatomy – with a more comprehensive map for the cognitive neuroscience model as illustrated in the figure at the top.

What have I learned about this so far:

1:  The pattern matching of yesterday is more complicated today – I taught a course in diagnosis and diagnostic reasoning for 15 years into the early part of this century.  Pattern matching and pattern completion was a big part of that course.  The patterns were fairly simple and involved visual diagnoses (diabetic retinopathy, rashes) comparing physicians at various levels of training.  The most dynamic aspect was the implication that experts were better at matching incomplete patterns than novices.  Today’s conceptualizations of knowledge structures and schemas contain concepts, actions, and dynamically alter what is retained in memory and what is not. 

2:  There are clear implications for psychiatric diagnosis -   the DSM classification and all of the criteria do not capture the reality of medical and psychiatric diagnoses.  There is a qualifier in the manual that it is not a substitute for experience but that is never defined.  That reason becomes a lot clearer looking these cognitive models.  Classification systems attempt to operationalize the diagnostic reasoning of a physician by averaging a verbal description of those events.  I don’t think that is possible and I will cite a couple of examples.

Example 1:  A psychotherapist refers a 27-year-old woman to a psychiatrist because of concerns that she has histrionic personality disorder.  She has not been able to make progress in therapy.  The psychiatrist seeing the patient knows within minutes that she is manic.

Example 2:  An intern is presenting the history of a 68-year-old man to his psychiatric attending.  The patient is extremely depressed to the point that he believes that he is cursed based on a trivial event that occurred in his childhood. Within the first 5 minutes the attending realizes that the patient is delusional and communicates that to the intern. The intern acknowledges that this is true and wonders how he failed to make that diagnosis.

Both cases highlight that knowledge of a classification system is not enough.  The psychotherapist and the intern both know the DSM and use it regularly. They have both had didactics in classification of mental disorders.  The only difference is that the psychiatrist in both cases has experienced cases of the disorder and had knowledge structures and schema to make the diagnosis.  Written descriptions of schema and knowledge structures are an incomplete approach to diagnostic reasoning. 

3: Classifications artificially separate actions from concepts – any reading of the DSM gives the impression that “this is the universe of psychiatric disorders – in order to function as a psychiatrist, pick one and then come up with a treatment plan.”  This is problematic at two levels.  First, if the cognitive neuroscience model of memories and knowledge structures is correct – a classification system is operating at a sublevel that averages features.  It is blind to the overall gist that despite this averaging no two people are alike.  Second, it removes action features that are necessary to function as a physician.  That would include top level schemas like “This patient is medically unstable and requires medical or surgical care first” or “This is a life-threatening problem that requires a safe and closely monitored environment." Some will argue that is not the goal of classification.  I would argue that many consider classification to be a diagnosis and in order for it to function that way – it needs to include action items in addition to a general rule out of causative intoxication states and medical problems. The DSM as it exists is classification without diagnosis.

4:  Cognitive neuroscience models highlight the fact that the separation between diagnosis and treatment is artificial.  All physicians are taught to do exhaustive evaluations of medical problems.  That is the initial step in a career.  It is also critical to learn when that exhaustive process needs to be immediately interrupted to focus on a more acute problem. I can still recall seeing a 7-year-old boy who have been hit by a car while playing in the street. He was alert but had significant abdominal pain.  The car bumper struck him just below his left rib cage.  It took me less than 5 minutes to determine that he had an acute abdomen and call the trauma surgeons. That non-linear process happens frequently in acute care psychiatry and in outpatient psychiatry with patients in crisis who need verbal interventions to assist in the diagnostic and treatment process.  

5:  Psychotherapy – there are recent perspectives on how cognitive psychology applies to the psychotherapeutic process at both the psychological and biological levels using these models.  Basically, maladaptive schemas are confronted and modified during the therapy.  There is some empirical evidence that this may happen particularly in the area of positive and negative self-schemas.  Much of this literature draws on existing cognitive behavioral therapy.  That leads to a question of what is the difference between a therapy focused on a cognition or an isolated memory compared with a schema focused therapy?

At the highest level of analysis memory focused therapies generally involve isolated autobiographical memories and schema focused therapies are about knowledge structures abstracted across multiple events that involve emotion, cognition, and behavior.  In theory the schema focused therapies may be useful in cases where the memory focused therapy is not effective, but a competing consideration is that schemas can be entrenched and difficult to change.  The memory focused therapy could be considered a bottom-up type of approach and the schema focused a top-down approach. 

6:  Criticisms – Criticizing the DSM as a diagnostic system is a cottage industry in the US and the UK.  As we approach a new version of the DSM expect most media sites to start months and even years of criticism. Practically everybody does it rarely discussing their motivations, understanding, and the limitations of their proposed system if they have one.  If diagnostic reasoning is a complex process consistent with the cognitive neuroscience models and requires direct experience, criticism of the manual rings hollow.  It is equivalent to reading about things that might exist and proclaiming you are an expert.  Psychiatrists with criticisms are also limited if they have insufficient experience in the areas they are criticizing.  Psychiatrists with the broadest experience will produce the best criticism. If you are criticizing a list of diagnostic criteria in a classification system in isolation – that is exactly what you are doing.  It is trivial compared with an actual diagnosis by a trained and experienced psychiatrist.        

This brief focus on the cognitive neuroscience of diagnosis should highlight that psychiatric education and practice is seriously lagging in this knowledge base.  If we are taking the “diagnosis” in DSM seriously it has to be modified to include this important brain science.  All of the current competing models face the same criticism.  A diagnosis by a physician is much more than typed criteria attempting to capture a dynamic process.  Secondly, psychiatry needs modern approaches to the mind. Approaches that correlate with neurobiology and have a clear empirical basis. Much of the DSM claims a sketchy atheoretical basis that should no longer be acceptable when powerful explanatory theories may exist.  Philosophy is no substitute.  Finally, we must find a way to implement these across all of our training programs and practitioners.  We should be devoting as many resources to integrating cognitive neuroscience into psychiatry as we do modifying the DSM.

And that should be the first step.  What does a DSM looked like with cognitive neuroscience baked in?  The answer goes a lot farther than “dimensions”.      

   

George Dawson, MD, DFAPA

 

 

References:

 

1:  Norman G, Young M, Brooks L. Non-analytical models of clinical reasoning: the role of experience. Med Educ. 2007 Dec;41(12):1140-5. doi: 10.1111/j.1365-2923.2007.02914.x. Epub 2007 Nov 13. PMID: 18004990.

2:  Brush JE Jr, Sherbino J, Norman GR. Diagnostic reasoning in cardiovascular medicine. BMJ. 2022 Jan 5;376:e064389. doi: 10.1136/bmj-2021-064389. PMID: 34987062.

3:  Custers EJ. Thirty years of illness scripts: Theoretical origins and practical applications. Med Teach. 2015 May;37(5):457-62. doi: 10.3109/0142159X.2014.956052. Epub 2014 Sep 2. PMID: 25180878.

4:  Koufidis C, Manninen K, Nieminen J, Wohlin M, Silén C. Unravelling the polyphony in clinical reasoning research in medical education. J Eval Clin Pract. 2021 Apr;27(2):438-450. doi: 10.1111/jep.13432. Epub 2020 Jun 22. PMID: 32573080.

 5:  Binder JR, Desai RH, Graves WW, Conant LL. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex. 2009 Dec;19(12):2767-96. doi: 10.1093/cercor/bhp055. Epub 2009 Mar 27. PMID: 19329570; PMCID: PMC2774390.

6:  Duff MC, Covington NV, Hilverman C, Cohen NJ. Semantic Memory and the Hippocampus: Revisiting, Reaffirming, and Extending the Reach of Their Critical Relationship. Front Hum Neurosci. 2020 Jan 24;13:471. doi: 10.3389/fnhum.2019.00471. PMID: 32038203; PMCID: PMC6993580.

7:  Insaustu R, Amaral DG. Hippocampal Formation. In: Mai JK, Paxinos G (eds) The Human Nervous System, 3^rd ed.  Elsevier, London, 2012: p. 933.

8:  Mazoué A, Gaultier A, Rocher L, Deruet AL, Vercelletto M, Boutoleau-Bretonnière C. Does a rabbit have feathers or fur? Development of a 42-item semantic memory test (SMT-42). J Clin Exp Neuropsychol. 2022 Sep;44(7):514-531. doi: 10.1080/13803395.2022.2133088. PMID: 36269845.

9:  Renoult L, Irish M, Moscovitch M, Rugg MD. From Knowing to Remembering: The Semantic-Episodic Distinction. Trends Cogn Sci. 2019 Dec;23(12):1041-1057. doi: 10.1016/j.tics.2019.09.008. Epub 2019 Oct 28. PMID: 31672430.

10:  Brown TI, Rissman J, Chow TE, Uncapher MR, Wagner AD. Differential Medial Temporal Lobe and Parietal Cortical Contributions to Real-world Autobiographical Episodic and Autobiographical Semantic Memory. Sci Rep. 2018 Apr 18;8(1):6190. doi: 10.1038/s41598-018-24549-y. PMID: 29670138; PMCID: PMC5906442.

11:  Teghil A, Bonavita A, Procida F, Giove F, Boccia M. Temporal Organization of Episodic and Experience-near Semantic Autobiographical Memories: Neural Correlates and Context-dependent Connectivity. J Cogn Neurosci. 2022 Nov 1;34(12):2256-2274. doi: 10.1162/jocn_a_01906. PMID: 36007071.

12:  Gilboa A, Marlatte H. Neurobiology of Schemas and Schema-Mediated Memory. Trends Cogn Sci. 2017 Aug;21(8):618-631. doi: 10.1016/j.tics.2017.04.013. Epub 2017 May 24. PMID: 28551107.

13:  Reyna VF, Edelson S, Hayes B, Garavito D. Supporting Health and Medical Decision Making: Findings and Insights from Fuzzy-Trace Theory. Med Decis Making. 2022 Aug;42(6):741-754. doi: 10.1177/0272989X221105473. Epub 2022 Jun 23. PMID: 35735225; PMCID: PMC9283268.

14:  Wilhelms EA, Fraenkel L, Reyna VF. Effects of Probabilities, Adverse Outcomes, and Status Quo on Perceived Riskiness of Medications: Testing Explanatory Hypotheses Concerning Gist, Worry, and Numeracy. Appl Cogn Psychol. 2018 Nov-Dec;32(6):714-726. doi: 10.1002/acp.3448. Epub 2018 Sep 1. PMID: 30686857; PMCID: PMC6345391.

15:  Hawke LD, Provencher MD, Parikh SV. Schema therapy for bipolar disorder: a conceptual model and future directions. J Affect Disord. 2013 May 15;148(1):118-22. doi: 10.1016/j.jad.2012.10.034. Epub 2012 Dec 4. PMID: 23218898.

16:  Lane RD, Ryan L, Nadel L, Greenberg L. Memory reconsolidation, emotional arousal, and the process of change in psychotherapy: New insights from brain science. Behav Brain Sci. 2015;38:e1. doi: 10.1017/S0140525X14000041. Epub 2014 May 15. PMID: 24827452.

Graphics Credit:

1:  The lead graphic as noted is from Cell Press and reference #12.  It is reproduced here with permission from Elsevier and this is their acknowledgement:

Reprinted from Trends in Cognitive Sciences, August 21(8), Gilboa A, Marlatte H. Neurobiology of Schemas and Schema-Mediated Memory, p. 618., Copyright 2017, with permission from Elsevier.  License 6278000229455, May 29, 2026 

2:  Second graphic was made by me using Microsoft Visio.

Supplementary 1:  Nobel Laureate and Psychiatrist Eric Kandel noted the importance of cognitive neuroscience years ago and this was a quote from his book:  The Age of Insight.

 

Medical Reasoning vs. A Diagnostic Manual

 

I taught a course on medical decision making and how not to mistake a physical illness for a psychiatric disorder from about 1990 to 2002. The main theorists at the time were all internists – Stephen Pauker, Jerome Kassirer, Richard Kopelman, David Eddy, and Harold Sox.  I read their papers and attended their courses.  State-of-the-art in those days involved extensive differential diagnosis, Bayesian analysis, and an awareness of an extensive list of potential cognitive biases. I had been impressed with the need for pattern matching and pattern completion and incorporated all those elements into my course.  I eventually pared it down to about 9 sections in the lecture notes illustrated with case vignettes.

My original emphasis was to recognize that there are several considerations when assessing the medical aspects of psychiatric care.  The first is the medical stability of the patient.  Can they be cared for on a psychiatric unit or do their medical needs require medicine or in some cases surgery?  Do they need referral to a generalist of specialist?  This is more complicated than it sounds because the patient is there seeing a psychiatrist for what is supposed to be a psychiatric problem.  But that presentation is complicated by several factors including most patients have no primary care physician and no routine health care maintenance. Many will come into the emergency department concerned about a medical problem but get sent to psychiatry. In that situation, people still get all of the acute medical illnesses including heart attacks, strokes, asthma attacks, pulmonary emboli, seizures, pneumonia, meningitis, encephalitis, and acute cholecystitis to name a few.  Many exhibit non-specific behaviors like agitation, crying out, aggression, or unresponsiveness that can be due to either a psychiatric disorder or a medical problem.    

The second is a psychiatric presentation of a physical illness in a communicating patient. The classic presentations involve brain pathology that is infection, inflammatory, vascular, trauma, or neurodegenerative.  Systemic endocrinopathies and inflammatory disorders are a close second. 

Finally, there is the patient with a clear psychiatric disorder who has intercurrent illness that is or is not known.  Examples that I have seen many times include current or new onset diabetes mellitus, profound anemia usually secondary to an upper or lower GI bleed, dermatology conditions that have often been neglected, symptomatic nutritional deficiencies (B12, folate, D), sexually transmitted diseases, complications of substance use like cirrhosis, and various acute and chronic infectious diseases.

Given that large population with diverse medical and psychiatric problems as well as diverse presentations that can include denying any physical problems – I typically reviewed how the diagnoses occurred.  Pattern matching was the fastest.  The physician has seen a physical finding, lab, behavior, etc – many times before, knows what it is, diagnoses it and treats it.  A good example is a rash.  Dermatologists are rash experts and can correctly classify rashes and marginal cases much faster than primary care physicians (4).  The same is true for diabetic retinopathy and ophthalmologists (5).  Until you have seen a person with severe mania or catatonia, neuroleptic malignant syndrome, or serotonin syndrome it is less likely that you can diagnose the conditions by reading criteria in a book.  Patterns are important for all medical specialists.

On the other end of the spectrum is the contemplative side of diagnosis.  There are several possible diagnoses, and it takes additional data, thought, and reasoning to come to a final diagnosis. Every medical student does this in their initial internal medicine rotation.  There is encouragement to produce a list of many diagnoses that might account for the presentation – but even as the case is being recorded or presented that list rapidly narrows to the apparent diagnosis.

In psychiatry, it may take much more data and collateral information to make a specific diagnosis at the initial presentation.  First episode psychosis (FEP) is a case in point. It is very important to determine what the symptoms onset was like and whether there were any associated mood symptoms or substance use problems. The patient may not be able to describe the phenomenology and depending on the circumstances treatment may be initiated while the diagnostic process is ongoing.  Teaching about the diagnostic process, we would spend time discussing what that might look like combined with a recursive approach to the patient and an awareness of cognitive and emotional biases.  I provided several examples of non-psychiatric physicians making errors due to emotional biases.

Since my course, the literature on medical decision making has changed to some degree.  There is some literature that addresses expertise in general at both the level of cognitive psychology (1) and neurobiology (2).  The general approaches have been to analyze expertise and diagnostic reasoning from the perspective of typical domains (cognitive, perceptual, motor) or to look at a general model and how that has developed over the years.

A dual processing model (3) is generally considered the best current representation of clinical reasoning and decision making.  In this model, there is a fast automatic, heuristic, and unconscious system called Type 1 and a slower conscious, analytical, and effortful system called Type 2.  Additional properties are indicated in the following table.

Parameter	Type 1	Type 2
Speed	Fast, automatic, unconscious/preconscious, little effort	Slow, deliberate, analytical, varying degrees of effort
Control	Minimum control, similar to automatic associations in everyday life except more focused	Control over thought process and direction
Systems and Processing	Pattern recognition and completion, implicit learning, access to long term memory	Working memory and manipulation of data in working memory, planning and reasoning based on that data
Memory Systems	Long term memory	Short term and working memory
Localization	-Orbitofrontal cortex (OFC) -Basal ganglia (caudate, putamen) -Insula -Anterior cingulate cortex -Amygdala -Hippocampus	-Dorsolateral prefrontal cortex (DLPFC) -Left inferior frontal gyrus -Middle frontal gyrus -Inferior parietal lobule -Precuneus -Hippocampus

 A clinical example of Type 1 reasoning is when a trained clinician recognizes a classic presentation of a medical illness, diagnosis, or finding.  An example I frequently use is when one of my Infectious Disease attendings who was an expert in Streptococcal infections recognized characteristic rash from across the room on a patient we were consulted for a different problem.  He made the diagnosis within seconds and told us how it could be confirmed.  In studies of the process the orbitofrontal cortex and limbic connections are activated.  Training is a critical element, especially seeing a maximum number of patterns and their variations.  Although the characterization is that this is a fast and automatic process, there is some room for deliberation.  For example, recognizing or attempting to classify equivocal cases without classic presentations. 

Type 2 reasoning is considered more of the typical process of differential diagnosis.  The findings are compared, analyzed, and accepted or rejected based on additional data and clinical judgment. This process is thought to localize in dorsolateral prefrontal cortex (DLPFC) the home of the working memory where data can be maintained and analyzed.  The left inferior frontal gyrus contributes to rule-based reasoning and hypothesis testing.  A clinical example from my experience is the case of the agitated stuporous patient.  These cases require a great deal of caution because they are most likely to represent a serious or life-threatening illness.  It requires a clinician who knows how to examine patients with stupor or coma and rapidly makes sense of the history and findings. It is a problem that can rarely be solved by Type 1 reasoning alone due to a fairly non-specific presentation.  Some of the critical points for hypothesis testing will be signs of increased intracranial pressure, purposeful response to painful stimuli, eye movements, reflex and musculoskeletal exam abnormalities, signs of infection, and meningeal signs.

The interaction between Type 1 and Type 2 systems is not necessarily sequential but it can be with the Type 1 system matching patterns that lead to hypothesis generation.  There is some evidence that in most clinical situations most of the diagnoses occur with Type 1 reasoning.  Experts can operate at the level of Type 1 reasoning due to extensive experience.  There is not necessarily a hard separation based on the properties in the table. Some hypothesis testing can occur at both levels.  Both systems are commonly grounded in both the limbic system and the hippocampus.

The human brain is capable of parallel distributed processing of data or information.  This means that there are many processing areas in the brain that are interconnected and they can all be working at once.  The modern conceptualization is brain networks that are active processing areas connected by white matter tracts widely distributed through the brain.  

That brings me to my model of diagnostic reasoning (see lead graphic and click to enlarge).  It is based on the course I taught, neuroanatomy and neurology, and what I have observed clinically. When I was talking about pattern matching 20 years ago based on my observations and reading studies in dermatology, ophthalmology, radiology, and pathology – the term seemed to fade rapidly from the diagnostic reasoning literature.  It was revived somewhat by the more recent focus on AI and comparison of that modality to humans.

There was a lull in Bayesian analysis after the invention of computerized programs like Quick Medical Reference (QMR) and Iliad.  They were designed to facilitate medical diagnoses by providing an exhaustive list of findings and their probabilities. These were 20^th century personal computer programs and not AI.  A study of these and 2 additional programs suggests that the programs got 52-71% of 105 diagnostic cases correct with 19-37% being the mean portion of correct diagnoses (6). Despite those figures the programs provided an additional 2 diagnoses per case that experts considered as relevant.  The authors recommended that the programs be used only by physicians who could include the relevant and exclude the irrelevant information provided by the programs.  The programs were discontinued without further modification or updates.  

That is the 8-mile-high view.  I plan to do a deeper dive into the neuroanatomy and neurophysiology.  But the clear reality of the situation is the ability to make a psychiatric diagnosis resides in the brain of a psychiatrist and not a classification manual or a checklist.   Manuals and checklists are crude approximations of some of the cognitive features that psychiatric experts possess.  Like all experts – skill will vary based on practice, exposure, and interest because of the effects on these brain systems.  But we are well past the point of equating what a psychiatrist does to a crude manual.  A manual never saved or treated anyone.  Further – the diagnostic reasoning process emphasizes elements that are important for education and training. It seems that in the past decades there has been a preoccupation with evidence-based research rather than the evidence itself. It does not do the physician or patient any good to be in a situation where that physician is unable to communicate with a person who is in a critical state and has no idea how to assess that problem.  Rearranging diagnostic criteria in a manual for the ninth or tenth time does not get you there.   

 

George Dawson, MD, DFAPA

Supplementary 1:   Before anyone says the diagram is too complex - it is a general diagram for any human diagnostician.  The main modifications for physicians and psychiatrists are the interactive aspects that include empathic comments, formulations, and numerous verbal interventions that other diagnosticians may not need to use.  The specifics about how these memory systems interact are not known at this point - I will be researching that over the next several months.  I borrowed the superposition concept from quantum mechanics - even though there are no wave functions for memory.         

 References:

.

1:  Bilalić M.  The Neuroscience of Expertise.  Cambridge University Press. Cambridge, United Kingdom. 2017.

2:  Maguire EA, Gadian DG, Johnsrude IS, Good CD, Ashburner J,  Frackowiak RSJ, Frith CD. 2000. Navigation-related structural change in the hippocampi of taxi drivers. Proc Natl Acad Sci USA 97:4398–4403.

3:  Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The Causes of Errors in Clinical Reasoning: Cognitive Biases, Knowledge Deficits, and Dual Process Thinking. Acad Med. 2017 Jan;92(1):23-30. doi: 10.1097/ACM.0000000000001421. PMID: 27782919.

4:  Federman DG, Concato J, Kirsner RS. Comparison of dermatologic diagnoses by primary care practitioners and dermatologists. A review of the literature. Arch Fam Med. 1999 Mar-Apr;8(2):170-2. doi: 10.1001/archfami.8.2.170. PMID: 10101989

5:  Sussman EJ, Tsiaras WG, Soper KA. Diagnosis of Diabetic Eye Disease. JAMA. 1982;247(23):3231–3234. doi:10.1001/jama.1982.03320480047025

6:  Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J, Baker AL, Ball EV, Cobbs CG, Dennis VW, Frenkel EP, et al. Performance of four computer-based diagnostic systems. N Engl J Med. 1994 Jun 23;330(25):1792-6. doi: 10.1056/NEJM199406233302506. PMID: 8190157.