Information versus Wisdom

I saw this post on another blog today and thought it was a good title.  I end up pondering this idea almost every day.  In medicine these days we are inundated by data scientists on the one hand and administrators on the other.  The data scientists tell us how they are going to revolutionize medicine through their analysis of large data sets.  The theory is that there are patterns in the data that can be detected only with advanced computational methods.  Having gone through the spreadsheet era and seen how easy it is to prove almost any theory with a large spreadsheet, I am very skeptical of Big Data.  Just dredging through the data, looking for patterns and writing it up does not seem very rigorous to me.  It strikes me more like one of the popular TV shows where the agents are in the field but solidly connected to the computer whiz back at headquarters who is capable of pulling up any document, any floor plan, and hacking into any closed circuit TV system in order to get the information that is needed.  I don't think that science works that way.

On the administrative side, it is the worst of times.  The statistical efforts of administrators are frequently laughable attempts to legitimize the next genius idea to come down the pike.  Their mistakes in healthcare are legendary ranging from the promise of the electronic health record to the RVU based management of physicians as widget producers, all exhaustively documented with numbers.  I sat in a meeting one day that showed 95% of the physicians in the department were not "producing" enough to cover their salary.  The problem was that nobody had done the multiplication on "work RVUs".  When the appropriate multiplier was added it was a different story.  Administrators also tend to collect a lot of numbers that they think will be useful for an analysis, without thinking ahead to the data analysis and statistics.  They seem to have no idea about basic statistical analysis much less more advanced analysis like how to legitimately analyze data over time to detect real differences.  There is no better example than the state of Minnesota collecting PHQ-9 scores over time from anyone trying to treat depression in the state.  They seem to think that unconnected collections of those numbers at different points in time will have some kind of meaning. Administrators also have the habit of creating studies that confirm their vision of the world and when those studies are complete - that is all of the "proof" that is necessary.  The entire concept of managed care rests on many of those studies.

On the wisdom side of things I can think of no better example than a colleague who I said goodbye to today.  He worked with me for the past 2 1/2 years.  He is an Internist who is also an Addictionologist and is ABAM (American Board of Addiction Medicine) certified.  He has been a physician since the early days of the HIV/AIDS epidemic and treating those patients was a significant part of his early practice.  He has an encyclopedic knowledge of the care of those patients and how it has evolved as well as being an excellent Internist.  He is interested in psychiatry and can talk in psychoanalytic terms.  He is also an expert in LGBT issues and can speak with authority on that subject.  I certainly did not want to see him go, but for the purposes of this post, I can think of no better example of wisdom that comes with medical practice.  He could be consulted on any number of complex problems in his areas of expertise and provide a very well thought out answer based not so much on information, but on what works and what the potential complications are.  Any physician can tell you that these are the folks you want to work with.  When I think about data mining approaches toward these areas of knowledge, I think about the 31 page document that is available online that looks at the issue of medication interactions of psychotropic and HIV medications.  It is a compulsively great document, but lacks the wisdom to help you pick the best therapy for a manic patient on tenofovir.

Granted my position is a thoroughly biased one.   I make no apologies for wanting to work with physicians who have the greatest technical expertise and know how to apply it.  I don't mean people who can recite facts or even algorithms.  I mean the people who know all of that and can look at the patient with the most complicated medical situation and still come up with a plan of action and how that patient must be closely monitored.  They also know when it is better to do nothing at all and that is a difficult skill to acquire.  Practically everyone leaves medical school and residency with a strong treatment bias.  You are taught to be "aggressive" and that most of the treatments that you do will do some good even if there is not cure.  In clinical practice, that is far from the truth.  In psychiatry for example, you have to recognize that there are certain biological predispositions, clinical patterns, boundaries, and personalities that are the warning signs of disaster with certain treatments.

When I first started in medicine in the 1980s, the wisdom based model was still the predominate model in most clinical settings.  Now it is much less frequent and there are departments that are just looking for people to fill in the gaps.  They don't necessarily want to retain you they just want to "keep the numbers up."  They also don't want you spending a lot of time on complex cases, because the payment rates rapidly decline if you are not shuffling people in and out the door.    When the administrators start recruiting bodies based on their revenue models and Hollywood accounting,  I hope that I will always end up on the side with the wisdom, rather than a heap of useless information.

There is a lot of that going around these days.  



George Dawson, MD, DFAPA      



Supplementary 1:  I was going to jam in a section to comment on emotional and moral reasoning in view of the expected backlash to the Rosenbaum articles in the NEJM, but decided to add it here instead.  It would have strained the above essay.  It has been an interesting (and fully expected) exercise in political rhetoric.  Predictably the critical articles mischaracterize her position and ironically are at least as guilty of the fallacies that they accuse her of using.  In one case, a new fallacy is pretty much invented.  I think it is instructive to note that in these matters, logic goes out the window.  There is no pathway to a sound judgment.  It basically involves rallying the troops to see who can shout the loudest.  My self proclaimed bias above is part of the reason I am firmly on her side (but will refrain from the shouting).  For anyone who thinks like me, there is no convincing me that the appearance of conflict of interest is the same as actual conflict of interest.  There is no convincing me that free pizza and donuts will cause me to blindly prescribe a medication - probably because I have not eaten Big Pharma food since the early 1990s.  In fact, if I think of a more plausible thought experiment about how much cold hard cash it would take to pay me off to prescribe a drug, I can't come up with a figure but it would have to be absolutely stratospheric compared with the usual speaker fees that people are listed in the Sunshine Act database for.  All of that is based on the fact that I work for a living and treat real patients.  I am accountable to those patients.  If a medication does not produce real results or it causes too many side effects (like my early experience with paroxetine) it is off the Dawson formulary and I don't prescribe it again.

This is of course like arguing with Democrats and Republicans.  I know that some pro-appearance of COI=COI will strongly dislike my experience and the way my thoughts on this matter are anchored in the way I practice medicine.  That is the nature of arguing about emotional and moral reasoning in what the Institute of Medicine (IOM) describes as an ethical vacuum.  The recent editorials certainly don't prove a thing.

The usual focus of these debates also leaves out the big picture that many entire University departments (math, science, engineering) actively collaborate with industries and in many cases actively invite industry participation in order to advance those fields.  The notion that physicians are not able to do that because they have a sacred trust to patients and would be somehow compromised remains implausible to me, particularly when nearly all of the major decisions that physicians make in this country have been seriously tampered with if not controlled by managed care companies and pharmaceutical benefit managers for nearly 30 years.

That is a massive conflict of interest that nobody talks about and it affects 80% of all of the healthcare in this country.

Supplementary 2: The graphic at the top of the post is from Shutterstock.

Neurotic Kids

I was watching the FX comedy program Louie (Season 5 Episode 5) and encountered one of the funniest scenes I have seen on television.  Louie is a neurosis based comedy, but it is also a show that many people will not be comfortable with because of content that results in the MAL warning.  In this episode Louie takes his 10 year old daughter in to see Dr. Bigelow played by Charles Grodin.  I heard that Grodin came out of retirement to play this character largely because he was impressed with Louis CK's technical expertise in filming the program.  We met him in an earlier season when he was trying to dispel Louie of the notion that he has done anything to get rid of his back pain and instead focuses on the philosophical predicament of the three-legged dog that he is walking.  After an introduction to the state of that animal he asks Louie: "What is the only thing happier than a 3-legged dog?"  I won't give away the lesson but you can find it on YouTube.

In this episode, Louie has brought his 10 yr old daughter Jane in to see Dr. Bigelow.  There are some preliminaries about whether she had a rash on her arm for 2 days or 30 days that has since cleared.  From there Jane goes on to consider: "Weird things in my head."  She is feeling like "I am sweating on the inside of my face" and builds this description with several "and then" clauses until she comes to a fantastic conclusion.  Dr. Bigelow looks at her and without skipping a beat gives her a response that I have both heard from physicians and takes care of the problem.  It also immediately shifts the frame from: "Is there something unusual about the way that Jane thinks?" to this being a completely acceptable exchange between a 10 year old and an old family doctor.  I am not going to disclose Dr. Bigelow's punch line for those who have not seen this episode and I encourage you to watch it.  It is worth it for this one scene that is so artistic, with timing so great, and it is the best acting from a child actor that I have ever seen.   It is incredibly funny.  I laughed out loud when I saw it and still laugh when I think of it.  Dr. Bigelow's comment is an example of the implicit message: "I am taking you seriously at the neurotic kid level and not commenting on your behavior like you are a little adult." It also caused me to reflect on my childhood as a neurotic kid.

Neurosis is an old word these days.  To me it always meant conflicted either in reality or at some symbolic level.  If therapists are involved, the conflicts end up being conceptualizations based on their theoretical models.  No matter how you cut it, anxiety is the common affect and there is usually a lot of it focused around unrealistic patterns of worry.  The child psychiatrists that I know dismiss many of the eccentric behaviors they hear about and are unconcerned about what a lot of parents seem very concerned about.  I have not assessed or treated children in over 25 years.  My work comes at the tail end of childhood neurosis.  The 18 year olds in high school and college students who become suicidal after their first boyfriend or girlfriend breaks off the relationship.  It has given me the opportunity to advise them why they are hurting and about life in general.  They seem to understand that by the time they get to my age that those problems in life will not hit them nearly as hard.  I reassure them that when that happens, meeting me will be a distant memory and I will probably be the only psychiatrist they will have ever met.

But it wasn't that long ago that I was a neurotic kid myself.  I won't disclose the full breadth of what happened to me so bear that in mind when you read about some of these incidents.  The first bad sign was that I have never really slept well.  Sixty years later that is still a problem.   I  am a chronic insomniac.  I also recall vivid nightmares as a kid, with frequent visits from a being I called a "Deathalow."  The Deathalow would just walk into my room at night and look very scary.  It was the kind of behavior you see in a lot of horror movies, so this is probably a common experience.  My parents and everyone else were puzzled because nobody had ever heard of a Deathalow.  But they finally caught a glimpse of the inner workings of my mind when I started pointing at Catholic nuns and screaming: "Deathalows."  Some time later, I pieced together the fact that Deathalows were a composite of a very bad chalk drawing of my grandmother's face in a nun's habit.

Just a few years later I was sitting in our living room watching television and I saw what appeared to be a Sir Walter Raleigh like figure walking up behind my father and preparing to stab him with a dagger.  I shouted out what was happening and my parents freaked out.  My mother was a frequent caller to our family physician and his advice was clear: "Stick him in a tub of ice water."  No visit to the ER to see the crisis team, just ice water.  They did so immediately, and while I was there I watched the comedy/tragedy masks on the walls in the bathroom laughing and crying while snakes slithered up toward the ceiling.  That was at least until I cooled off.   Then all of the hallucinations vanished.  But it was the death of a family member that was all I needed to develop the longest preoccupation that I had in childhood - death and physical illnesses.  For a while I was preoccupied with having cancer, rabies or being poisoned.  I recall one incident after a Soviet nuclear test when we were warned about a large cloud of fallout passing over northern Wisconsin.  We were advised to stay indoors.  At the peak of that fallout, I can recall seeing radioactive particles floating in the air.  The rabies preoccupation was the longest.  I played football almost every day and was always alert to the presence of dogs.  At one point, I thought that a dog may have had rabies and I had inoculated myself with the virus after I fell catching a pass.  For months, I monitored myself for the development of symptoms of rabies.  I would get up several times a night to look in the mirror to see if my physical appearance was changing (I was up anyway).

Around this time, I started to get nightmares about a large glass pyramid.  There were several tiers of panels in the pyramid and on each panel was the face of a woman wearing Kabuki make-up shouting in a shrill voice: "Chinese ghosts!".  In each case, I would wake up extremely anxious and wonder why I was dreaming that dream.  And then... one night I decided that I really did not have to walk into that pyramid.  It had a very long entrance-way.  I thought before I fell asleep that night: "Just wake up if it looks like you are going into the pyramid,  You don't have to go into that pyramid."  And I was right.  I woke myself up before the entrance to the pyramid and it was gone.  I never dreamed that dream again.  But the neurotic behavior in the daytime was harder to get a handle on for a long time.  I had to tell myself that I had no control over if I lived or died.  In some cases, I got some very negative feedback on the poisoning hypothesis as in: "Are you accusing me of poisoning you?"  I eventually forced myself to think of other things.   Eventually that forced aspect was gone as I developed more interests.  As my reading and research in other areas increased, my worries about cancer, rabies, and death dissipated.

Throughout all of this, I never saw a counselor, therapist, or psychiatrist.  I got the "Dr. Bigelow advice" from our family physician with treatments ranging from "throw him in a tub of ice water" to a rather primitive creosote-like nasal lavage that all of the kids in my family got if we went in to see him for a cold.  I am convinced it was an aversive therapy to keep us out of his office.  I have never seen that treatment used anywhere else in medicine.  

This merely scratches the surface of my experience as a neurotic kid.  It may be why I got such a laugh out of Dr. Bigelow's advice.  And of course it also causes me to wonder what would have happened if I had received psychotherapy or medication for these "symptoms."  Would I have encountered one of the wise child psychiatrists I know or somebody who thought I was psychotic?  I was definitely not as calm about it back then as I am recalling it now - there were after all snakes on the walls!

But I eventually turned it around on my own and became a guy who can appreciate the humor in being a neurotic kid and somebody who can relate to them.


George Dawson, MD, DFAPA




Supplementary 1:  No guarantees on how you will find Louie.  I find much of his comedy brilliant, but some is also cringeworthy so as always watch at your own risk.  The segment I am talking about is less than 2 minutes long about 2 1/2 minutes into Season 5 Episode 5.

Supplementary 2:  To all my psychiatric colleagues out there, I did think about these disclosures.  Hardly anybody reads this blog and I don't anticipate doing any transference based psychotherapy.  I think it is also pretty obvious that you reach a point in your life where all of these neurotic behaviors are irrelevant.  All of the other main players are dead or forgotten and there is no emotional impact.  The experiences themselves are history and have been for 50 years.  That is how I chose the disclosures.  In part they were also modeled on some disclosures I have read in books written by psychiatrists who disclosed things that happened to them as adults.

Supplementary 3:  The more I reflected on the historical context of neurosis, the more I realized that it means something different now than when I was a kid.  When I was a kid, it meant that you were crazy in the popular sense of the word.  Nobody had a nuanced appreciation of mental illness and how anxiety or obsessions were different from psychotic disorders.  Today, I think neurotic behavior is reinforced to a point.  For example, the parents who say: "He or she is 12 going on 30" and seem to see their children as small adults who may need some competitive advantage like cognitive enhancement.

Supplementary 4:  The glass pyramid graphic is a download from Shutterstock for non-commercial use only and this is a non-commercial blog.
 
Supplementary 5:   A useful interview question for adults with anxiety and depression:  "There are all kinds of theories about how people get anxious.  One of those theories is that our minds come up with stories to fit the level of anxiety that we have.  That can be transmitted from one person to another.  Looking back on your childhood can you recall anyone who seemed to transmit their anxiety to you?

The NIMH Director and the RDoC - The Politics and The Science

from: Insel TR, Cuthbert BN. Medicine. Brain disorders? Precisely. 

Science. 2015 May1;348(6234):499-500.

I caught this article about the RDoC criteria for classifying mental illnesses based on various non descriptive parameters and neuroscience in the journal Science a couple of weeks ago.  As any reader of this blog can attest, there is no stronger advocate for the role of neuroscience in current psychiatric practice and the future of psychiatry than me.  There has been media controversy on this subject and it is always difficult to determine how much real controversy exists and how much of it is just made up for the sake of media self promotion like much of the DSM-5 controversy was.  Reading through the article by Thomas Insel and Bruce Cuthbert  there are statements that can be taken at face value.  I think these statements are consistent with the position that clinicians in general are not very scientific and are also outright clueless in some areas.  This is a bias that I have certainly heard from other scientists and it does not serve the cause of science very well, especially if the goal is to advance neuroscience and bring everyone up to speed on that discipline.  Dr. Insel has presented his view that all of the trainees in the clinical neurosciences of psychiatry, neurology, and neurosurgery should rotate through a year or two of a shared neuroscience.  When I first heard him present it five years ago I thought it was a great idea.  In the time since and especially after getting a response from him, I think it is less clear.  It would be great if every department of psychiatry had neuroscientists on staff to teach neuroscience.  But they don't and there is also the problem of neuroscientists being focused on research rather than teaching.  On the other hand, there are plenty of bright people in those departments who know a lot about the brain.  It is a question of reconciling these two points to come up with the necessary infrastructure yet in this article the authors make it seem as if large clinical problems are not addressed and that clinicians are fumbling around with very crude assessment methods.

They list three articles as examples of the RDoC.  The most interesting of these articles is one from the American Journal of Psychiatry that proposes that computer abstracted data from hospital notes that is converted to RDoC criteria are better predictors of hospital length of stay (LOS) than DSM criteria.  Just considering that method my first impression was that there was a lot wrong with that picture.  First of all,  LOS data is tremendously skewed based on non-clinical practices.  All it takes is hospital case managers with some success in intimidating physicians to skew the data in favor of business rather than actual medical or psychiatric discharge decisions.  Second, the quality of data from inpatient settings is incredibly bad due to the toxic combination of electronic health records and government billing and coding regulations.  As a reviewer, I have seen thousands of inpatient records, some of them hundreds of pages in length and I have found EHR records are notoriously poor in information content.  And finally, I thought the RDoC was a new system designed to be dependent more on neuroscience than the DSM-5?  How does methodology that looks at this DSM biased, sketchy clinical data result in a RDoC diagnosis?  Looking at the graphic from the Science article at the top of this post, it is pretty clear that 3 out 5 data dimensions under "Integrated Data" are basically clinical data.  There is a smugness displayed in the report similar to what might be seen in a rant by an antipsychiatrist: "For now clinicians might be best advised simply to be aware of the usefulness of dimensional models to capture psychopathology."  and "This result should provide some reassurance to clinicians that their notes do contain relevant detail for deriving dimensional measures of illness; like Molière’s Bourgeois Gentlemen speaking prose without knowing it, clinicians may already speak some RDoC."

Really?

The average person I see has chronic insomnia and has had possible sleep terrors and nightmares in childhood along with social phobia.  At some point they developed either severe anxiety or depression, but they can't recall the sequence of events and they currently have both.   They typically think that they have had "manic episodes" and may have been diagnosed with bipolar disorder even though they don't know what a manic episode is.  All they know is that their symptoms have persisted usually without remission for the past 10 to 15 years.  Of course that is complicated by the fact that they have been using marijuana, alcohol, and opioids in excessive amounts since then,  they may not have a significant family history of psychiatric and addiction problems, and they have the expected childhood adversity and adult markers of psychological trauma and abuse.  Further, I know from talking to the same people in repeated initial evaluations over the years that they don't give the same history twice and rarely remember much about their medications or psychotherapy treatment.  Should I use a "placeholder diagnosis" (pejorative term from reference 4) or should I assume that I am dealing with the social phobia that the patient may have had in childhood?  The idea that an RDoC diagnosis is going to give me an answer to that question any better than a DSM-5 diagnosis is pure folly if you ask me.  At least until we get the promised neuroscientific markers promised by the NIMH.  In fact, the description of the RDoC in these articles is reminiscent of another technology that was supposed to diagnose mental illness and that was quantitative EEG or QEEG.  I know quite a lot about QEEG, because I purchased a machine in the 1980s after a promising article on the technology came out in the journal Science.  I researched it using highly skilled EEG techs and an expert in neurophysiology to run the protocols, and concluded the diagnoses that came from the computerized analysis of the tracing were no better than chance in terms of what patients presented with.  Like RDoC diagnoses, the computerized analysis of QEEG data was highly dependent on the input of clinical data collected by the clinician.  It allowed the clinician to add and subtract clinical variables and look at how the diagnosis varied.  

The staff and researchers at the NIMH need to decide if a superior and critical attitude toward physicians who use current clinical approaches and are successful with them is the best one.  It should be obvious from the above analysis that many of us are not as naive or as ignorant about science as they expect. My proposed solution would be a more collaborative approach including the following:

1.  Recruit and train neuroscience teachers - most of them are already out there.  For example much of what I teach to trainees interested in addiction and addiction medicine is neuroscience.  It is also much more realistic than waiting for every department to have access to neuroscience researchers and then expecting those researchers to teach in addition to doing research.  My guess is that every Psychiatry department already has faculty that teach neuroanatomy, pharmacology, brain science and neuroscience already and that most of them are not officially scientists.

2.  Make the reading list available online - the article refers to over 1,000 published articles that focus on the RDoC criteria.  These should be available though the National Library of Medicine web site along with other neuroscience articles of interest to psychiatrists.  An added bonus would be CME activity available for self study.

3.  Post a list of neuroscience modules and build on that list -  In a previous post, I posted two links to neuroscience modules through the NIMH.  I would put up two lists, one containing a growing list of modules and the second with a list of the neuroscience concepts that need to be illustrated.  This would be useful for psychiatrists, psychiatrists in training, and medical school professors hoping to make their basic science lectures more relevant, since many clinicians still seem to have difficulty understanding how neuroscience is important in psychiatry.

4.  Better graphics - make high resolution graphs that illustrate detailed brain anatomy and basic science available online for teachers.  Pulling this material together is often the most difficult part of the teaching job and it requires an intensive effort to not run afoul of copyright laws.   It would be easier to recruit neuroscience teachers if there are high quality teaching materials available.

5.  A neuroscience teaching blog - In addition to the NIMH staff posting the references, concepts and modules, an open teaching blog should also be available.  I would encourage it to be a platform for discussing concepts and how to present them to trainees.  Ideally, it would be a place for active dialogue about the concepts and teaching them.

I think that all of these measures would be helpful in building an infrastructure of neuroscience teachers, neuroscience teaching, and a mechanism for the widespread dissemination of this material in residency programs and in educational programs for practicing psychiatrists.  If the RDoC is in fact worthwhile, there is plenty of brainpower outside of the NIMH to figure that out.

It is the brainpower that is currently focused on coming up with solutions and resolving problems of incredible clinical complexity.  And that happens every day.

I plan to send these recommendation to Director Insel and see what he thinks.



George Dawson, MD, DFAPA

1: Insel TR, Cuthbert BN. Medicine. Brain disorders? Precisely. Science. 2015 May 1;348(6234):499-500. doi: 10.1126/science.aab2358. PubMed PMID: 25931539.

2: Casey BJ, Craddock N, Cuthbert BN, Hyman SE, Lee FS, Ressler KJ. DSM-5 and RDoC: progress in psychiatry research? Nat Rev Neurosci. 2013 Nov;14(11):810-4. doi: 10.1038/nrn3621. Review. PubMed PMID: 24135697.

3:  NIMH.  Research Domain Criteria

4:  McCoy TH, Castro VM, Rosenfield HR, Cagan A, Kohane IS, Perlis RH. A clinical perspective on the relevance of research domain criteria in electronic health records. Am J Psychiatry. 2015 Apr;172(4):316-20. doi: 10.1176/appi.ajp.2014.14091177. PubMed PMID: 25827030.

Supplementary 1:

The above figure is licensed through the American Association for the Advancement of Science - license number 3637270124183.

Minnesota Finally Rejects Managed Care

In a rather stunning reversal of thirty years of public policy, the Minnesota Legislature voted last week to fund for a "network of small treatment centers" to compensate for all of those years of rationing.  The details are still vague  but it is described in the StarTribune article by Serres as a "network of small treatment centers, to be built across the state, anchoring a broad package of preventive services so children don't end up in emergency rooms or inpatient psychiatric wards where many of them are discharged prematurely for a lack of beds."  The funding is about $13 million for 2 years and $6.6 million "to create a network of 30-bed treatment centers for children with highly aggressive or self-injurious behaviors, who are often turned away from hospital inpatient units."  Political speak is evident in that sentence.  I can't imagine that $6.6 million or even $19.6 million gets you a lot of 30-bed treatment centers.  Later in the article a total new bed capacity is described as 150.

The author of the article is oblivious to how this all happened in the first place.  Was Minnesota an idyllic place with no mental illness in the first place?  Did the problem arise because an epidemic of mental illness?  Absolutely not.  Thirty years ago, the state had more bed capacity and more treatment options for both children and adults with mental illness.  Minnesota is a state with massive managed care presence and those managed care companies currently run all of the acute care psychiatric beds in the state - for both children and adults.  Once managed care companies learned that they could deny hospitalizations based on some fictional "dangerousness" criterion and otherwise ration psychiatric services at multiple levels there was - in effect - no rational psychiatric care.  As I have posted on this blog many times, people are discharged from hospitals in a few days essentially without treatment, treatment units are chaotic without a therapeutic environment, people who require medical detoxification form drugs and alcohol are generally out of luck, and it creates and ongoing demand on emergency departments and correctional facilities.  All it took was getting these practices embedded in the state statutes and setting up a cursory review of complaints against managed care companies at the state level to seal the deal.  The reporter in this case makes it seem like the Minnesota Legislature and a "bipartisan coalition of lawmakers" are solving a problem.  This is a problem they created in the first place and I will believe in a solution when I see it.

I have some first hand knowledge of the problem with children's services from my contact with child psychiatrists around the state.  Their experiences are echoed in the story in this article of what happens when your child is out of control and nobody is willing to help.  The family in this case describes a 17 year old boy with severe mood problems, aggressive thoughts and thoughts of self harm.   He was hospitalized 6 times and discharged in a heavily medicated state.  He was turned down by 30 residential facilities before being accepted by an out of state facility.   His parents describe themselves in a "perpetual state of anxiety" trying to manage all of these scenarios.  But the most incredible line in the article:

"For years, children who exhibit highly aggressive or violent behavior in Minnesota have been forced to drift from one short term hospital to the next, often returning to their families heavily medicated but with their illnesses largely untreated."

This is not surprising to any psychiatrist.  This is the end product of managed care rationing and it occurs whether the patient is a child or an adult.  It happens when businesses and governments collude in providing some bastardized version of psychiatric care.  It happens when psychiatrists in this case are ignored.  When state officials ignore psychiatrists. When psychiatrists who are trained to treat aggressive and violent behaviors are not allowed to do their jobs.  After all, why would anyone with aggressive behavior because of a mental illness be turned away from a psychiatric unit?  Aggressive and suicidal behavior are the main reasons that psychiatrists exist today.  It is what we do.  Let us do our jobs.

So far this is one small victory for children's mental health advocates and my cap is tipped to them.  But to reverse more of the problem we need to acknowledge what it is and it is managed care or more specifically their marketing word for mental illness - behavioral health.

Let's get rid of it entirely.

   
George Dawson, MD, DFAPA

Reference:

Chris Serres.  Facing chronic shortages, Minnesota's mental health system gets a boost.  Minneapolis Star Tribune, May 29.  


Supplementary 1:  The image used for this post is of Dexter Asylum attributed to Lawrence E. Tilley [Public domain], via Wikimedia Commons.  The original image was Photoshopped with a graphic pen filter.

Supplementary 2:  For a detailed post on some of what happened try this.

Physicians Replaced By Computers - Lessons From A Roomba

My Memorial Day project was purchasing a Roomba and getting it up and running.  I am a big believer that robots will make all of our lives easier at some point and decided now is the time to start walking the walk.  For those not familiar with the Roomba, it is designed as a robotic vacuum cleaner.  Once you have set up the rooms and programmed it, it is basically supposed to vacuum your floors automatically and then park itself in a docking station for charging.  The machine itself is about a 14 inch diameter disk that rises to a height of about 3 1/2 inches off the floor.  It is a light 8.4 pounds.  It is able to accommodate sharp angles with a secondary brush that spins on an arm that extends from under the main disk.  This combination of the main disk spinning and the extended spinning brush cleans the corners of a room.  I purchased the latest model, a Roomba 880 after consulting with friends and relatives who had earlier models.

One of the considerations in buying the Roomba was whether it would help turn my home into an even cleaner environment than it currently is.  That is a tough act.  One of my friends who is a physician gave his opinion that my home is "museum-like".  My office is probably the only problematic room with stacks of books and journals piled everywhere.  Disarray certainly but minimal dust.   My entire first level is hardwood flooring that is typically vacuumed with a built in system.  To its credit the Roomba contains all of the debris in the machine until it is emptied and all of the exhausted air is HEPA filtered to avoid exhausting any dust particles.  The main cleaning mechanism consists of two debris extractors that are rubberized bars that spin at a high rate of speed across the floor surface to capture dust, hair and larger particles.  But the most interesting aspect of the Roomba was going to be its observed behavior.  It has two modes when vacuuming.  It can start in a spot and spin increasing circles in an outward direction until gets to about a 3 foot diameter and then it spirals back in to the center spot.  In the more typical mode it heads to the room perimeter and then "automatically calculates the room size and cleaning time."  The most valuable tip in the manual was to take measures to restrict it to one room at a time and it comes with two Virtual Wall^®Lighthouse^TM devices that allow for easy demarcation of the work area.

The most fascinating aspect of getting started with the Roomba was going to be setting it up and watching how it went about the task of vacuuming.  I did some very minor room preparation, charged it up, and turned it loose.  As expected it headed straight for a wall and then attempted to establish the perimeter.  I remembered this as standard rodent behavior.  If you have ever confronted a mouse in an open area of your floor, their first move is to dash to the baseboards and run parallel to them to escape.  That strategy works well in the wild because the maneuver is associated with more cover and makes them less susceptible to predators.  It works much less well when confronted by a human who knows that it is their first move.  And yes, scientists have bred mice that do not exhibit this behavior.  My guess is that they would not fare well if they made it outside the lab.  The Roomba's behavior is less rigid than a typical mouse with some exceptions.  In the hour and 20 minutes it took to vacuum the adjacent kitchen and great room - it circled a kitchen island perfectly at least 10 times, but at the wall perimeters it was much less predictable.  At times the Roomba would peel off and take off across the room in a single pass or rarely return and continue along the original wall.  Sometimes it would head off the wall at a 45 degree angle and at other times 90 degrees.  There were never the usual adjacent passes that a human would make using a standard vacuum cleaner.

According to the literature,  the Roomba is supposed to "crisscross" the room in order to clean the floor.   I placed two small pieces of popcorn in the middle of a large section for flooring to use as markers of cleaning efficiency.  In the course of an hour, the Roomba passed these markers many times, sometimes very closely without vacuuming them up.  During that time it was very difficult to detect how much crisscrossing had occurred since mouse-like it spent the majority of the time in the periphery, bumping and spinning around walls and furniture.  It eventually did break free from the walls and set off on a 45 degree path picking up one popcorn fragment at about the one hour mark and the other at about one hour and ten minutes.  The old adage about pictures is true and I happened across this 30 minute time lapse photo of the Roomba working a room (with permission from the SIGNALTHEORIST web site).  It correlates well with my description of the actual paths.

As I surveyed the job afterwards, the floor was definitely clean and the warning light on the machine was saying that the dustbin was full.  When emptied, it contained an impressive amount of debris and dust relative to what seemed visible to the naked eye.  Another win for the robots?  Well, not really.  It is an interesting tool that I will continue to use and study, but in comparison with humans it is not efficient and at this point certainly not autonomous.  Despite all of the guidance in the manual the Roomba can still encounter unpredictable surfaces and get itself into trouble.  In my case it was the pedestal of a recliner.  The wood at the edge was about 3/4 of an inch high.  For some reason, the machine did not recognize it as an obstacle and continued to run up the base and get hung up.  A loud spoken error message would sound advising the human in the room to pick it up and start it in a new place.   The other concern is efficiency.  It spent far too much time in the perimeter and a low percentage of time covering the main floor areas.  That was tremendously inefficient.  It took at least 5 times longer to vacuum the main rooms than I would have if I was pushing a vacuum cleaner.  Even though it allows me to do other things, it says something about current state of available and affordable domestic robots.  They can't match the performance of humans on a fairly basic task.  This is an important concept.

Before any of the futurists out there jump on me for being a Luddite, let me disclose a few details.  I am a member of the IEEE and have been for the past 18 years.  I am currently a member of the IEEE Robotics and Automation Society.  I am not an electrical engineer and I have not designed or built any robots, but hope to start doing this when I retire from psychiatry.  I consider myself to be an expert in the human brain and the advantages it confers on humans over other animals and machines.  The Roomba is a basic case in point.  It cannot sense and adapt to novel conditions quickly enough to match a human doing the same task.  Even more striking is that although it is designed to vacuum homes and I have a fairly typical home with a better than average floor surface, it still encounters situations that exceed its response capacity.  In those situations it needs a human assist.  What is it about the human brain that leads to that kind of an advantage?  First and foremost, it is a rate of pattern matching and pattern completion capacity that allows us to recognize vacuuming problems, anticipate them and correct them by developing novel solutions even before the problem leads to a stop in action.  Some of this happens when a human goes around the room to set it up for the first time for the Roomba.  That human has made some assessment of the machines capacities and limitations and is problem solving for the machine before it is turned on.

Observing the limitation of the Roomba leads me to a point where I can address both the idea of computers replacing doctors and how that fits into the common anti-physician narrative in this country.  Is there a connection between the two?  My experience tells me that there is.  For nearly 30 years there has been a constant stream of antiphysician rhetoric.  The sources have been expected.  One of them is the key opinion leaders (KOLs) of the managed care industry.  I can recall reading one of the the first books written by one of them, a non-physician who was widely acclaimed as being an expert in managed care.  His early theory was that the high cost of health care was due to the decisions that physicians make.  But in the middle of the book he wrote what he thought of physician salaries and only grudgingly acknowledged that they should probably be paid a good wage due to their education.   I have posted here many times my experience at a managed care conference in the 1990s.  The speakers at that conference were very clear that the explicit agenda of their industry was to replace all of the specialists with primary care physicians.  The examples given were orthopedic surgeons and psychiatrists.  When a psychiatrist in the crowd pointed out the shortcomings of that philosophy - he was called a "whiner" by a Governor who was an anointed KOL in the industry.  Then the KOLs from the financial services industry started weighing in.  You could find glimpses of it while reading the investment literature.  People who were investors with no particular degree started saying that some day, physicians would get what they deserve - with the implication being that whatever that was - it was not good.  Any physician has experienced this prejudice.  The comments about how physicians are "expensive" as a rationalization for working them to death by not hiring any additional help.  Replacing physicians with computers seems like a logical extension of this rhetoric.  Googling this topic returns a number of provocative articles written from a point of view that is generally consistent with who the author is.

I know that some of those authors know the difference between a robot or a computer and a doctor, but it is also clear that some do not.  They certainly don't seem to understand that the real processing power of a human diagnostician's brain is in the area of pattern matching.  In order to duplicate that property with current technology, takes a massive computer and it is one of the reasons why my new $700 Roomba, although well designed - can easily be beaten by a human with a standard vacuum cleaner.  But the human advantage goes far beyond that.  Human diagnosticians do far more than match simple patterns.  They are able to complete fragments of patterns and anticipate what the whole pattern should be.  For example, is it likely that a depressed person is in this current state as the result of an inherited form of depression, their current state of detox from an opioid and/or benzodiazepine, current stressors or interpersonal conflicts, brain trauma, an undiagnosed medical condition, childhood adversity, psychological trauma as an adult, or defects in reasoning at either the emotional or cognitive levels.  Then there is the matter of acquiring all of the data to make the determinations.  Patterns upon patterns of data.  The Roomba-like approach would be to give the person a checklist of depressive symptoms and pretend that is all that needs to be known.  Checklists are already being administered by a computer and may be administered by robots someday.  

Yet it takes the pattern recognition, and several layers of it, as well as human experience dependent learning in order to make a real medical or psychiatric diagnosis.
        

George Dawson, MD, DFAPA






Supplementary 1:  The graphic at the top of this post is a photo that I shot of the inside of the box that my Roomba came in.

Supplementary 2:  I don't want to give the false impression that I do a lot of vacuuming.  My wife does practically all of it, but I am trying to do more especially if there is a high tech twist to it.  Some of the first robotics I hope to work on will be human controlled arms and hands designed to do yard work and move heavy objects around in the house.  I can't believe this is an area that has been ignored.

Moral Bias

Lisa Rosenbaum's final installment of her three-part conflict of interest series in the NEJM is out and full text is available online for free.  My associations and observations in response to the first two can be found here and here.  As a student of bias and rhetoric, reading Dr. Rosenbaum's series has been a breath of fresh air both on its own merit, but also relative to the grim anti-industry and anti-physician bias that permeates the popular press and medical settings these days.  The hypocritical nature of many of these comments was always obvious to me, but there was only very qualified support for anyone who did not see all physicians (especially psychiatrists) as tools of the pharmaceutical industry.  But the types of moral and ethical biases that Rosenbaum highlights goes far beyond the issue of free lunch from a pharmaceutical rep.  It shakes the very foundation of a system of evaluation based on weak empirical evidence or pure politics as I have pointed out many times on this blog.  At some level it is such a stunning expansion of many of the old NEJM editorials, it seems surrealistic that these articles have been published.  But on the other hand, the page below this last article invites readers to participate in a poll on the ".....suitability of three potential authors to review articles for the Journal."  Combined with the fact that these articles have a series editor that suggests to me that this may be all part of a social media-like initiative to attract interest to the NEJM.  On that basis, I expect a full gamut of future authors including the more typical opinions equating the appearance of conflict of interest with conflict of interest and suggesting zero tolerance for contact with industry.

This article starts out with commentary on the medical school "anti-pharma animus".  The organization of American medical students apparently grades medical schools on the basis of how free from pharmaceutical company influence their conflict of interest policies and environment are.  She gives a quote from a medical student to illustrate the mind-set involved in at least some members of this movement, namely the need for "pure" information to medical students.  Some early critics of Rosenbaum's article cite this as anecdotal data but that misses the point.  Her point is that this kind of mind-set exists and it is one of a number of mind-sets that makes the ethical climate around conflict of interest an unreasonable one.  She also points out that the rhetoric associated with this statement clearly indicates that this is a moral argument and at that point the psychology of moral arguments may apply.  From the perspective of medical education, is it better to take an insular approach and suggest that all research can be assessed by looking at the funding source or should medical students be taught to read and critique research independent of funding source?  A study quoted one of Rosenbaum's previous article suggests that internists are able to look at research abstracts and classify them according to research rigor, but that the introduction of funding source forces a re-evaluation with a bias against industry funded research.

The article progresses to talk about the psychology of moral argumentation at that point and a set of arguments that I have summarized in the table above.  I think it is also instructive to address one of the early arguments about the Rosenbaum essays and that has to do with evidence and the use of evidence in arguments.  In order to look at that, it requires a quick look at the type or argument, whether it is a scientific or non-scientific argument and whether the corresponding type of evidence exists.  I think there is no doubt that Rosenbaum's arguments are moral arguments rather than scientific ones.  As such they seek to address the ethical climate around conflict of interest.

There are two aspects of the concept of ethical climate that are missing from Rosenbaum's analysis.  The first is the ethical climate as a way to control physicians.  The best example during my career has been managed care and the research that supported it.  Like today's collaborative care research, the early managed care research was focused on the idea that it was more "cost-effective" than fee-for-service or treatment as usual.  In both cases (collaborative care and managed care), the research was generally done by advocates of the proposed methods.  Thirty years later, any objective analysis on the effect of managed care on psychiatric services will show that it has been devastatingly negative.  Bed capacity has been shut down, the criteria for inpatient care is "dangerousness" rather than any specific medical indication, people are clearly discharged from hospitals based on optimizing meager DRG based payments rather than medical indications, detoxification and addiction services have practically been eliminated from most hospitals,  only a small percentage of hospitals have psychiatric services, state hospital systems have also been shut down, and the only place where psychiatric care has increased has been county jails and prisons.  That entire system wide change for the worse was based on a moral argument of cost-effectiveness rather than scientific research.  Once that ethical landscape was established physicians could simply be shouted down with the slogan: "Times have changed - you are no longer in charge."  I doubt that any physician who heard that slogan was ever in charge of anything.  It was political rhetoric, designed to elicit an emotional reaction in the people taking over and the physicians they ultimately came to control.  Nobody thought that cost effectiveness was the same thing as cost shifting to correctional systems.

The second aspect is the explicit control of physicians by managed care companies and a conflict of interest that greatly exceeds that of any other industry.   Rosenbaum's three articles are all focused on the pharmaceutical or medical device industries.  There is no mention of the managed care industry or its spinoffs, despite the fact that it controls the medical care of over 80% of Americans.  As I have consistently pointed out, the theoretical concerns of the affiliations of authors on research papers about drugs or medical devices is nothing compared with a managed care company that tells your physician that you need to be discharged from a hospital or use a particular medicine that the physician is not recommending.  In the case of psychiatric care, that company is free to make even more life altering decisions such as denying a patient with a drug addiction any functional detox services, deciding that a patient with significant suicide risks can be treated on an outpatient basis, or maintaining a person in a disabled state with minimal treatment options for a complete recovery.  The regulatory environment that concentrates that much power in an industry that can generate profits by denying care is a complex story, but it all started with an ethical environment  that blamed physicians for the high cost of health care.  That physicianscold environment has too many elements in common with the pharmascold environment to ignore.  In both cases there is a predominate moral bias that greatly oversimplifies the problem and at least in the case of managed care leads to clear long term adverse consequences.              

The good news is that these articles have been published along with the evidence that moral reasoning can be seriously flawed and associated with biases.  Rosenbaum's focus has clearly been on the relationship between physicians and the pharmaceutical or medical device industry.  She has discussed her personal experience as a Cardiologist and how it has affected her largely in terms of interventions, statin therapy, and as a potential consultant to the industry.  Psychiatry has been an easier target for the same biases and rhetoric that she lists in her article.  I pointed it out in a Washington Post article where the narrative was clearly skewed to fit the idea that psychiatry was corrupted by Big Pharma and attempting to make it easier to diagnose depression in order to sell more antidepressants.  That article included selected information to make it seem like the American Psychiatric Association was the only professional organization to make advertising profits from Big Pharma.  The suggested quid pro quo for advertising revenue should be absurd to anyone familiar with advertising but it was not to this reporter.  But the real issue was that the DSM does not recommend treatment anyway and the majority (80%) prescribers who treat depression don't use a DSM-5 or even care about what it says.  Less formal approaches adopt a similar scolding moralistic tone toward psychiatry that is possible only by ignoring the deficiencies in other medical fields and idealizing them while devaluing psychiatry.

I think that Rosenbaum's articles are must reads, especially for psychiatrists who may be unfamiliar with rhetoric, moral reasoning, and politics.  That may be why physicians in general have been inept in mounting any kind of a counterattack against political strategies that work by changing the ethical climate.  These articles provide some points for discussion.   Watching the counterattacks will also be instructive.      


George Dawson, MD, DFAPA

1: Rosenbaum L. Beyond moral outrage--weighing the trade-offs of COI regulation. N Engl J Med. 2015 May 21;372(21):2064-8. doi: 10.1056/NEJMms1502498. PubMed PMID: 25992752.

Bias Cuts Both Ways

Lisa Rosenbaum's second article (1) in a series of three appeared in the New England Journal of Medicine this week.  It continues the theme that bias is more complicated than following the money (or pens or pizza slices).  It was interesting to note the response to the original article.  On at least one blog a poster apparently Googled Dr. Rosenbaum in order to point out all of her potential conflicts of interest.  It probably would have been more relevant to look at the standard International Committee of Medical Journal Editors disclosure form on file at the NEJM.  But it does provide a good example of one of the references in this week's piece - financial disclosure (or in this case suspicion) as an ad hominem approach to evaluating science.  In a similar vein, one of the authors of a previous NEJM editorial pointed out that the science has to stand on its own merit independent of disclosures or credentials.

The initial part of the article points out a fact that has always seemed pretty obvious to me.  Association is often used to suggest that there are significant meaningful conflicts of interest.  The oft quoted statistic is that 94% of physicians have "relationships with the industry".  This includes counting drug samples or donuts in the workplace as a significant relationship.  One of my previous employers decided that all hospital employees should participate in "donut rounds" and not just the physicians.  I viewed this as a pseudo-egalitarian managed care tactic, but by definition that means that 100% of the employees at that hospital had a "relationship with the industry".  The article also points out that physicians who request drugs be included in a formulary are also more likely to have industry relationships than not.  I have a history of being on two separate Pharmacy and Therapeutics (P&T) Committees of a major healthcare system and a major hospital where the overriding biases were drug cost and deals with pharmaceutical companies.  There were major changes on a year to year basis due to price differences in the cost of commonly prescribed drugs and they were often based on the flawed assumption that all drugs in the same class are equivalent.  On the other hand if an exclusive deal could be made, that company might also be able to combine that with a preferred deal for a more common drug.  It was a curious situation where decisions that were supposed to be science based were not.   The P&T Committee was biased by the managed care industry operating strictly on a cost basis.

I decided to part ways with the P&T Committee on the day we were presented with forms to fully disclose all of our finances to assure there were no conflicts of interest.  The usual hype about how we were all cherished guardians of the public trust was included.  I thought about it for a second and looked around the room.  There were approximately 15 MDs and 4 PharmDs on the committee.  Some of the brightest and most well read staff in the organization.  I ran the hypothetical through my mind: "What are the odds that if I owned stock in XYZ Pharmaceutical that I could convince my 20 colleagues that it was a good idea to put their most expensive drug on the formulary just based on my word?  What are the odds that I could overcome the predominate bias of the financial well being of the company that employed all of us?  What are the odds that if I accomplished those first two unattainable goals that I would actually see a profit in my XYZ Stock?"  I also thought about the drug approvals that we did based solely on public relations considerations.  Drugs that were practically worthless, but that were demanded by advocates for incurable diseases.  The explicit decision was that we did not want to run afoul of some of the very vocal advocates for those illnesses.  It would be bad public relations and press.  I looked back at the 5 page disclosure form and decided that I was not going to play that game.  I was not going to pretend for one second that I needed to be vetted by an organization that apparently saw itself as destined for sainthood.  I walked away at that time.  I had long been familiar with the other bias listed in the Rosenbaum article and how any disclosure at a CME presentation could result in mockery or discreditation.  It is difficult enough to present the pertinent information to an audience who may want to be entertained without being the butt of jokes as a sponsored presenter.  Any way you look at it my behavior was a reaction to a modern day witch hunt mentality and I was not about to be declared a witch.  Even beyond that, I was not the only "clean" person in the room.  Getting heavy handed about a conflict of interest disclosure as a public relations gimmick after many of us had already figured it out 20 years ago was more than a little insulting.

Rosenbaum also discusses a previous study from the NEJM (2) that as far as I can tell has not received press anywhere else.  In this study, the authors designed identical fake clinical trial report abstracts of studies with varying methodological rigor.  The studies had three possible funding disclosures - the pharmaceutical industry, the NIH or none.  In this experiment the internists involved in the study were less likely to read the entire report, less likely to prescribe the study medication, and less likely to see the experiments as rigorously conducted if they were sponsored by industry rather than the NIH.  They were half as likely to prescribe medications recommended in an industry sponsored trial than an NIH sponsored trial.   The authors were naturally concerned that physicians would be skeptical of even high quality but industry sponsored trials and that might slow the acceptance of that information in clinical practice.

In an interesting study done by the industry that same year,  a team (3) at the biotechnology firm Amgen  examined 53 studies that were considered to be 'landmark' preclinical studies in the field of oncology.  The goal was to see if they were reproducible.  The findings could be confirmed in only 6 or 11% of these papers.  The non-reproducible research led to hundreds of secondary articles and in some cases led to research that exposed patients to agents that resulted directly from the original research.  A team (4) at Bayer HealthCare did a similar review of preclinical studies to identify potential drug targets and concluded that only 25% of the work was reproducible.   That same article referenced the general industry rule that up to 50% of published results from academic settings cannot be reproduced in an industrial lab.  The Amgen and Bayer HealthCare studies suggest that is really an underestimate of what amounts to academic bias.  A common criticism leveled by Big Pharma critics is that they have benefited from all of the taxpayer funded basic science research at the NIH.  These studies suggest that there are plenty of problems with that research independent of funding stream and that they are less rather than more likely to lead to drug discovery.

In addition to unique information to examine the issues of bias (the author lists many more),  she also points out the origin of the word Pharmascolds (5) as those who "vilify the medical products industry and portray academics working with it as traitors and sellouts".  The authors here describe an inadequate public response from both the companies and the physicians who were criticized.  The full text of the reference is available online.

This is another highly informative article by Rosenbaum which demonstrates that analyzing conflict of interest is not as easy as some members of the press and some professional critics make it out to be.  She gives a great example of an 85 year old woman who needs an aortic valve replacement and what the considerations might be based on the characteristics and biases of two different Cardiologists assessing this situation.  It is even more complex in a setting where some would be quick to refer the woman to a hospice service.  In my experience outlining all of the potential biases in that situation is a significant task and one that few physicians would attempt.

Some day a more measured discussion of conflict of interest might provide a better approach.  Until then be skeptical of any critics who suggest that it is inappropriate for a physician to have a relationship with the pharmaceutical or medical devices industry just based on the face of it.  And for any physicians out there who want to take the most conservative path to avoid unwarranted criticism, stay out of the Sunshine Act database.

        


George Dawson, MD, DFAPA        
                   

1:  Rosenbaum L. Understanding bias--the case for careful study. N Engl J Med. 2015 May 14;372(20):1959-63. doi: 10.1056/NEJMms1502497. PubMed PMID: 25970055.

2:  Kesselheim AS, Robertson CT, Myers JA, Rose SL, Gillet V, Ross KM, Glynn RJ, Joffe S, Avorn J. A randomized study of how physicians interpret research funding disclosures. N Engl J Med. 2012 Sep 20;367(12):1119-27. PubMed PMID: 22992075.

3: Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012 Mar 28;483(7391):531-3. doi: 10.1038/483531a. PubMed PMID:22460880.

4:  Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov. 2011 Aug 31;10(9):712. doi: 10.1038/nrd3439-c1. PubMed PMID: 21892149.

5:  Shaywitz DA, Stossel TP.  It's time to fight the 'pharmascolds'.   Wall Street Journal.  April 8, 2009.    http://www.wsj.com/articles/SB123914780537299005 accessed on May 17, 2015.