The Documentation Fallacy

"If it isn't documented it didn't happen!"

That is the documentation fallacy in a nutshell.  At first it seems like an obvious truth.  A silence falls on the crowd, everyone looks at the floor, and we move on.  Fallacy accepted.  I have seen the scene play out a thousand times, scripted by unimaginative attendings.  It is also scripted by administrators and attorneys who have a lot more invested in the process.  I can still recall a malpractice scenario in one of my throw away journals in residency.  A malpractice attorney walks in to depose an internist.  The patient in question is a diabetic who has lost his right leg to gangrene.  The attorney is questioning the doctor about wound care provided to the patient.

Attorney:  "Reading from your notes doctor what leg did you treat when you saw the patient."
MD:  "I treated the patient's left leg."
Attorney:  "Are you absolutely sure.  Did you document treatment of the left leg?"
MD:  "Yes I am sure - you can read it right here in the notes."

Case closed.  The attorney was hoping for no right or left designation in the original chart and an easy malpractice settlement.

This is a powerful vignette about why documentation needs to occur and how it can be protective in terms of risk management and avoiding malpractice litigation, but is it really that simple?  To take a look at the fallacious aspects of that statement requires an examination of what I call the period of excessive and useless documentation.  I will provide a couple of anchor points.

The first is my neurosurgery rotation in both my third and fourth years in medical school.  I was on a very busy neurosurgical service at Froedtert Memorial Hospital in the early 1980s. The hospital was brand new and there was a question of what services would be located there,  Neurosurgery and Neurology occupied an entire floor.  The team consisted of two senior neurosurgery residents, two general surgery interns, a general intern, and a medical student.  The residents spent a great deal of time in the operating room with the staff neurosurgeons and efficiency was critical to the entire operation.  We had to round on 20-30 intensive care unit (ICU) and general beds and discuss it with the residents by 10AM in a conference room.  All of the daily documentation had to be done by that time, because all day and night long there were calls to the emergency department (ED) and the ICU.  The ED consults involved a brief walk over to Milwaukee County Medical Center - the next building to the east (in those days) on the grounds.

A standard hand written progress note on a non-ICU patient on this service in those days was "Afebrile, VSS, wound looks good - no signs of infection."  We of course checked all of the wounds, labs, vital signs and did other focal exams as necessary.

Flash forward to just before 911.  I am sitting in a conference room with colleagues from my multispecialty group. We are listening to a presentation by a billing and coding specialist on all of the bullet points that are necessary to complete a note.  The examples shown are notes of about 300 to 500 words in length.  We are told that unless all of the bullet points are ticked off or commented on we could be prosecuted for billing fraud.  Not only that, but if a "fraudulent" bill goes out in the mail we could be prosecuted for mail fraud and possibly conspiracy under the anti-racketeering RICO statutes.  There had been several high profile prosecutions of health care organizations and individual practitioners with FBI involvement at the time.  We were told that our healthcare organization at the time now had an internal compliance bureau that would audit all of our notes to make sure the bullet points were checked of to avoid the large multimillion dollar fines and of course jail time.  A racketeering charge could result in federal prison time.  Clinical notes used by physicians had suddenly been usurped for an entirely different purpose - legal leverage by government agencies and businesses.  That leverage is used to deny payment, ration services, and generally exhaust physicians so that they don't have time to fight these tactics.

Flash forward a third time.  The year is 2009.  I am now sitting in a large multispecialty committee meeting on documentation and hospital oversight.  We are given several hospital progress notes that are 16-18 pages long.  That is a single progress note from one day that is 18 pages long.  We are told that several physicians are routinely compiling notes this long.  I say compiling because the electronic health record being used allows physicians to rapidly pull data in to the note from many places in the chart to rapidly build the note.  It also allows physicians to build their own custom templates and phrases to add to the note.  The note looks terrible because it is a mix of fonts and spacing - a great example of the primitive state of the electronic health record (EHR) that persists right through to today.  All of the notes are designed to meet billing criteria determined by the federal government rather than demonstrate contact with an intelligent life form.  EHRs - even fabulously expensive ones seldom produce a coherent, readable document and may even spread that incoherence over a ream of paper if you ask for the records.

That has been the progression of excessive and useless documentation as I have experienced it in my career.        

But here's the reality.  If I spend 60 minutes with a patient and don't tick off some bullet points that don't really apply to what I am doing - don't think for a minute that "if it isn't documented it didn't happen."  Try that experiment yourself.  Sit down and have a one hour conversation with a friend and then decide how you are going to document what happened.  I have been a student of Communication Theory since I read the first paper by Shannon And Weaver and and have never seen an adequate discussion of optimal information transfer between two people or how it should be recorded.  The only way to get to the content would be a verbatim recording or transcript and that would contain a lot more information than is typically contained in one of my notes.  I could try to approximate that by writing an 18 page note but let's also assume that like most doctors you have 5 - 10 minutes to document something.  It is obvious the vast majority of communication that happens will not be documented.  In psychiatry there is the added issue of people who say: "I want to talk about this but I don't want you to put it in the medical record."  That information is generally very sensitive and in some cases is considered privileged psychotherapy information separate from the medical record and unavailable to other providers.  It is still a question of what gets documented, but what is documented is still information depleted relative to the original conversation.

Now - let's consider what the US government and by default most insurance companies want physicians to do.  For a standard outpatient assessment of varying intensities there are a number of  "bullet points" required to meet billing criteria.  That means that a certain number of them need to be checked off.  If they are - the bill can be submitted.  There are huge quality problems with that approach.  I previously posted the questions that I ask about sleep to practically all of the patients I see for evaluation whether they have a primary sleep compliant or not.  On a lot of outpatient forms a sleep complaint is a single check box.  On the most widely used screening tool by managed care organizations for their collaborative care approach - the single question is:  3. Trouble falling or staying asleep, or sleeping too much.  The choices are "not at all", "several days", "more than half of the days", and "nearly every day".

Think about that sleep screening question for a moment.  It is important because a lot of managed care clinics have it right in their electronic health record.  The patient may check it off on a tablet and it is imported into their record  on an ongoing basis.  After all of that whiz bang technology what do we know?  We know that the patient has one of three sleep problems (even a physician with a paper form could circle one and immediately upgrade the quality of information).  And we know approximately how many days per week the problem exists.  Go back to my sleep questions and compare the information content.  And yet these managed care settings are highly likely to have somebody sitting in a meeting, looking slightly annoyed and endlessly voicing the Documentation Fallacy and the importance of these checklists.  In the case of the questionnaire and many if not all template approaches - it was documented and you don't really know what happened.

In addition to poor quality, low information content, reduced direct patient contact time, and excessive time taken to generate - notes that are designed for billing and administrative purposes are also a drain on the environment. On current hospital medical records systems  they take up disc space.  Not as much as imaging data, but when you look at the graphic at the top of this page - all of these low quality, information poor notes are piling up by the tens of thousands every day.  Contrary to the traditional use - for relevant historical data and to learn what previous physicians were thinking - we currently have exabytes of data that is so information poor it is generally never seen again.  It was viewed once by a billing and coding specialist and once by an insurance company and then it is banished to one of the storage arrays that are running 24/7 - never to be read again.  More importantly - never to be read by a doctor again.  Thirty years of stakeholder meetings got us here in the first place.  If physicians cannot finally assert themselves - the profession will continue to do this scutwork till the end of time.

So when you hear the Documentation Fallacy uttered - feel free  to gasp and roll your eyes.    



George Dawson, MD, DFAPA



Supplementary 1:

The American College of Physicians is the only physician professional organization to take a stand on the unnecessary administrative burden placed on physician in the United States.  That is a very recent position and a departure from the usual positions taken by professional organizations that physicians should be prepared to fall into lock step and do whatever documentation that governments or insurance companies or electronic health record manufacturers want them to do.  You can read Putting Patients First by Reducing Administrative Tasks in Health Care: A Position Paper of the American College of Physicians at this link.  It is obviously a politically correct approach that is basically a call to stakeholders.  Stakeholders with a conflict of interest like looking for any excuse to not pay or pay less will not be motivated to change 30 years of what amounts to physician exploitation.

A quote from that position paper:

"Related work by Sinsky and colleagues (46), also discussed earlier, focused on how physician time is allocated in ambulatory care and found that physicians spent 49.2% of their time on EHR and desk work, versus 33.1% on direct clinical face time with patients and staff."


Supplementary 2:

I posted this a while ago on some additional documentation that psychiatrists were supposed to do about quality to avoid payment penalties.  If you follow the link Physician Quality Reporting System you will be taken to the APA web site.  Note in the right column a heading called  View the list of 2017 MIPS Individual Quality Measures.  A click on that link leads to a long list of various "quality measures" and how to report them.

Supplementary 3:  

I have not had time to go through all of the documents on the APA web site but a number of them are written by the NCQA and NQF.  Neither of these organizations would be considered as quality initiatives by physicians.  In my opinion, the NCQA started as a managed care heavy organization and I am sure any objective analysis of the outcome measures would illustrate that.  The NQF started by political mandate and I am sure carries forward the usual political biases of all of the self declared health care experts sitting in Congress.  So how do they end up as further reasons for more documentation by physicians who are cranking out so much paperwork that they have no time to see patients?  And how do they end up on the web page of a physician professional organization?

A telling statistic from the NQF web site: "30% of NQF endorsed measures are developed my medical specialty societies."  Where do the rest come from?

Attribution:

Graphic at the top is from Shutterstock per their standard licensing agreement.  Credit is
"Stack of the old paper documents in the archive." by Loginova Elena.

Interviewing 101

Interviewing seminars are a big part of the first year of psychiatric training.  I am not sure how it goes these days, but I can recall having to record interviews and being critiqued by the instructor and all of my peers in seminars.  I can remember not always agreeing with the critiques.  Every psychiatrist goes on to develop their own interview style around the basics.  Just about every interview is unique because it also depends on the person you are talking with.  The flow of information in the interview has always been fascinating to me.  At times you can cover all of the essential elements in 30 minutes.  At other time you can talk for 90 minutes and end up with 25% of the information.  Facilitating and directing that flow of information is one of the key elements of interviewing.  Against that backdrop I found this commentary on an interview of Robert Downey, Jr. somewhat interesting.  It seems surprisingly linear.  It reminded me of some of my media interview experiences where the predominate advice seemed to be: "No dead air.  Either I am talking over you or you are talking over me.  Got it?".  This clip has been widely broadcast for the past several days.  In it, the interview takes a bad turn and Downey politely gets up and walks out.  The discussion near the end of this brief clip suggests that there were probably just a few minutes left.







In the critique of this interview, Kathleen Kelley Reardon focuses on what is described as the human chemistry between the participants and how that potentially involves features like attractiveness, mood, timing and other features.  Reardon speculates that Downey may not have been up for the interview and the transition to personal questions may have been premature. She sees it as an excellent case study in what can go wrong with interviews.  I think that there are some good examples of what might go wrong but there are also some unknowns.  From my vantage point as a psychiatrist I have a few other observations and I have never had to worry about the tone of the interview, but then again I am never working on an interview as an infomercial.

1.  The introduction is very important, but we may have missed it.  Even though this is supposed to be the entire interview, it begins with Downey speaking.  This is an old Oral Boards style point - if you don't introduce yourself and set the context of the interview - come back and try it again next year because you have just failed the exam this year.  The interviewer could have saved himself a lot of problems by discussing the interview context ahead of time and setting ground rules for what the actor is or is not willing to discuss.  You don't have time to discover that in an 8 minute interview.

2.  Too many people in the room is never a good thing.  Because I am teacher, I still have people observing my interviews for teaching purposes and I never like it.  I have to be completely focused on the other person and how they are affecting me.  I did not see Downey as distracted or disinterested; I saw him look to his advisor several times until it got to the point he was overtly looking for advice.  I saw him make a clear announcement at one point: "Are we promoting a movie?" where he was clearly dissatisfied in the direction things were headed and that comment was directed to other people in the room.  The best way to maintain focus is to make sure that there are only two people in the room.

3.  What is the purpose of the interview?  There has to be a focus on that point and the interviewer needs to be aware of it.  In a psychiatric or medical interview the overriding agenda is that there is a mutual focus on a problem that needs to be solved for the patient.  Everything is as confidential as possible.  I heard a prominent psychiatrist and researcher say at a psychotherapy conference that some of the primary goals are: "Be nice to the patient and say something useful to them."  In a celebrity interview there are really dual agendas - publicity for both the interviewee and the interviewer.  Being a celebrity interviewer can lead to celebrity status on its own.  The interviewer is probably aware of how they want to come across to the viewers.  Where do they want to be along the famously provocative to famously uncontroversial spectrum?  Do they aspire to be a celebrity interviewer?  How focused are they on entertainment versus journalism?  I personally cannot think of a greater intrusion into the interview process.

4.  Contrary to the author's point, I don't think that the reporter (Krishnan Guru-Murthy) had a problem with transitioning or failing to read the cues of Downey. He seemed anxious to me. Downey came across as authoritative when talking about American cultural influences but then somewhat oppositional and defensive when talking about a past opinion that he gave during an embarrassing period in his life. He was aggressive when commenting on the interviewer's motor behavior and suggesting that he was running out of time to ask (what was probably going to be) a controversial question.  This would have been an entirely different interview if the focus had been maintained on superhero culture and the actors theories of where the film fits into that genre. He had a pretty good interpretation of some of the Stan Lee origins in Vietnam era America.  Just the time line of those developments and the further implications for the film would have filled the time.

5.  Sometimes the person being interviewed drops a gift at your feet and you have to go with it. As an example, if I am interviewing a person who has been incarcerated I rarely go directly after that information.  I can probably get the historical details elsewhere and it is a threat to the interview process.  I don't want the interviewee to develop the "cop transference" and start to experience it as a police interrogation.  And I usually have an hour compared to this less than 10 minutes session.  The  interviewer needs to be aware of the fact that he is not doing psychoanalysis and that all of those Barbara Walters interviews where there was a key emotional disclosure occurred after hours of interviewing and heavy editing.  In this case Downey talks about how he portrays the character and how his interpretation of the character had changed over time.  That leaves him talking to a small part of the audience for this interview - the people interested in art and acting but that would have ended a lot better.

6.  As I watched the interview, I had the question about whether there were any journalistic biases operating.  It becomes clear that Guru-Murthy wants Downey to answer questions that have nothing to do with the movie and were from a very difficult time in the actor's career.  It is clear that the reporter's anxiety level is building as he tries to force those questions.  And, it is clear that he is trying to force them into the smallest possible window in this interview - the final minutes.  It was anxiety provoking for me to watch that section of the interview.

7.  There is often a lot of focus on the process aspects of the interview.  It seems that the emphasis on the communication aspects of the interview are very linear - pick up this cue and make this intervention.  Interviews (at least the way I see them) are non-linear,  There are a lot of parallel processes going on and interviewers tend to elicit much different information based on their biases and techniques.   There may be times where I slow the interview way down to get at specifics and at other times I am looking for global markers and whether they are present or not.  

8.  Based on my past experience,  I also had to wonder if the gotcha dynamic was operative.  I have been called in for media interviews where the reporter has some preconceived notion of how the world works.  A good example is the fallacy that the Christmas Holiday season is the peak time of the year for suicides.  After I had spent some time explaining to the reporter over the phone that this is really not true, during the interview I was pummeled with comments and anecdotes about how people naturally get depressed and kill themselves more often during Christmas.  This has happened to me more than once and it is a good reason to avoid reporters.

The way this interview ended seemed quite civil to me.  It is not surprising to me that the media is making a big deal of it in spite of the fact that really catastrophic interview endings tend to occur with people who are accusatory, demanding, threatening and/or aggressive.  In an interview with an actor that is not likely to happen.

Despite all of our focus on interviewing in psychiatry, we seem to be loathe to look into the science of it all.  For the past 30 years we have been operating under the illusion that in order to make a DSM diagnosis, all it takes is getting the answers to the right questions.  Those questions were typically structured interviews using DSM or the precursor RDC criteria.  It gave way to the Diagnostic Interview Schedule (DIS) for early epidemiological work followed by the Schedule for Affective Disorders and Schizophrenia (SADS).  This work seems to have led to brief diagnostic checklists based on  the DSM criteria.   I read an article in the Journal of Clinical Psychiatry once that suggested if all of the clinicians in a clinic used the SADS as their diagnostic interview they would have better outcomes.  The idea that a structured interview or checklist elicits better or more useful information than an experienced psychiatrist interviewing the patient is another great fallacy in the field.  I would actually put that at the top of the list and rate it higher than needing a head to head comparison of antipsychotics based on time to discontinuation or whatever the Cochrane Collaboration has to say about "limitations of methodology / need more study" for practically any drug trial.  The evidence that I am right is replicated tens of thousands of times every day by psychiatrists out there doing the same work.  If you interview the same patient twice it is very unlikely that they will give  you the same history.  I have a standard flashcard that lists about 100 medications of all classes and they will not consistently endorse the same medications on this list.  We interview people about their subjective experience and that experience is always plastic.  That is much more interesting than storing the encyclopedia on computer chips.

In some cases we might put a metric like test-retest reliability on an interview metric or the global result of a structured interview.  Given that we are measuring something that reflects the functioning of a highly plastic organ, I don't know why we would expect reliability to be high.

That brings me back to this interview.  Our interview technology is a holdover from the 1950s.  We have evolved subtle modifications over the years but currently we are constrained to a small fraction of the conscious state and we do not know how to optimize the flow of relevant information.  This is a major limitation.  There have been some theorists who have looked at mapping diagrams of the interview process but none have gained any widespread acceptance.

The only good news for psychiatrists now is that we are not operating at the level of reporters.


George Dawson, MD, DFAPA

Big Data and Psychiatry - Moving Past the Mental Status Exam

I was a fan of big data before it became fashionable.  I was a high tech investor before the dot.com bubble and became very interested in high speed networking, especially the hardware necessary to move that data around.  Even before that information was publicly available, electrical engineers were using that equipment to rapidly download large amounts of data (GB) from satellites on every orbit.  As an investor, one of the early flagship applications was large telescopes.  I wrote an article on high speed networks and the medical applications - digital radiology and medical records back in 1997.  At about the same time I made the information connection.

As a college student, I got my hands on the Whole Earth Catalog.  That led me to my small college library and my surprise to find  that they had Shannon's  seminal work on information theory on the shelf.  I was even more excited when I learned about entropy in my physical chemistry course three years later.  Since then I have been searching without much success to look at what happens when two people are sitting in a room and talking with one another.

My entire career has been spent talking with people for about an hour and generating a document about what happened.  It turns out that the document is stilted in the direction of tradition and government and insurance company requirements.  It covers a number of points that are historical and others that are observational.  The data is basically generated  to match a pattern in my head that would allow for the generation of a diagnosis and a treatment plan.  The urgency of the situation can make the treatment plan into the priority.  The people who I am conversing with have various levels of enthusiasm for the interaction.  In some cases, they clearly believe that providing me with any useful data is not in their best interest.  Others provide an excessive amount of detail and as the hour ends I often find myself scrambling to get to critical elements before the hour expires (my current initial interview form has about 229 categories).  This basic  clinical interview in psychiatry has been the way that psychiatrists collect information for well over a century.  In the rest of medicine, the history and physical examination has become less important due to advances in technology.  As an example, it is rare to see a cardiologist these days who depends very much on a detailed physical examination when they know they are going to order an echocardiogram and get data from a more accurate source.

In psychiatry, other than information from a collateral interview and old records  there is no more accurate source of information than the patient.  This creates problems when the patient has problems with recall, motivation, or other brain functions that get in the way of describing their history, subjective state, or impact on their life.   The central question about how much useful information has been communicated in the session, the signal-to-noise considerations, and what might be missing has never been determined.  The minimal threshold for data collection has never been determined.  In fact, every information specialist I have ever contacted has no idea how these variables might be determined.

Information estimates have become more available over the past decade ranging from estimates of the total words spoken by humans in history to the total amount of all data produced in a given year.  Estimates of total words ever spoken range from 5 exabytes to 42 zettabytes depending on whether the information is stored as typewritten words on paper or 16-bit audio.  That 8,400 fold difference illustrates one of the technical problems.  What format is relevant and what data needs to be recorded in that format?  The spoken word whether recorded or typed is one channel but what about prosody and paralinguistic communication?  How can all of that be recorded and decoded?  Is there enough machine intelligence out there to recognize the relevant patterns?

An article in this week's Nature illustrates the relative scope of the problem.  Chris Mattmann makes a compelling argument for both interdisciplinary cooperation and training a new generation of scientists who know enough computer science to analyze large data sets.  He gives the following examples of the size of these data sets: ( one TB = 1,000 GB)

Project	Size
Encyclopedia of DNA Elements (ENCODE), 2012	15 TB
US National Climate Assessment (NASA projects), 2013	1,000 TB
Fifth assessment report by the Intergovernmental Panel on Climate Change (IPCC), due 2014	2,500 TB
Square Kilometer Array (SKA), first light due 2020	22,000,000,000 TB per year

That means that the SKA is nearly producing the total amount of information spoken by humans (recorded as 16-bit audio) in recorded history every year.   The author points out that the SKA will produce 700 TB of data per second and within a few days will eclipse the current size of the Internet!

All of this makes the characterization of human communication even more urgent.  We know that the human brain is an incredibly robust and efficient processor.  It allows us to communicate in unique and efficient ways.  Even though psychiatrists focus on a small area of human behavior during a clinical interview the time is long past due to figure out what kind of communication is occurring there and how to improve it.  It is a potential source of big data and big data to correlate with the big data that is routinely generated by the human brain.

George Dawson, MD, DFAPA

Dawson G.  High speed networks in medicine.  Minnesota Physician 1997.

Lyman, Peter, H. Varian, K. Swearingen, P. Charles, N. Good, L. Jordan, & J. Pal. 2003. How Much Information? Berkeley: School of Information Management & Systems.

Mattmann CA. Computing: A vision for data science. Nature. 2013 Jan 24;493(7433):473-5. doi: 10.1038/493473a.

Shannon CE.  A mathematical theory of communication. The Bell System Technical Journal 1948; 27(3): 379-423.