Will AI Be Better Doctors or Will It Kill Us All First?

 

AI never leaves the headlines.  At least two leading AI CEOs claim that AI technology will eliminate all human diseases within the next decade.  There is some suggestion that they do not have a very good understanding of human disease since one referred to cancer as a single disease.  Other people reacting to those headlines have suggested that will not happen. I think it is highly unlikely.

The medical literature is becoming less controversial.  A recent opinion piece in JAMA (1) suggests that AI surpasses not only physicians but also AI assisted physicians in at least 5 cognitive tasks of physicians ranging from data acquisition to prescribed therapies.  The authors suggest that by 2030, AI alone will consistently produce better diagnostic and therapeutic results than physicians.  They do qualify that opinion in several ways.  First, the test comparisons are generally not clinical examples but more like test problems.  Second, they point out that AI alone cannot exceed physicians performance in radiology.  That finding was quite surprising until I discovered they were referring to procedural radiology rather than just image interpretation.  Third, the literature is affected by publication bias – negative AI studies are apparently not published.  And fourth, there are barriers to implementation but apparently not enough to keep the authors from concluding:

“Nonetheless, superior autonomous AI will likely be ready to be deployed for real-world cognitive medical tasks in some, maybe many, workflows by 2030.”

Let me digress for a paragraph about science fiction and the pitfalls of AI. Science fiction writers have been warning about this for decades.  Kubrick’s classic 2001 A Space Odessey came out in 1968.  One of the central features in the film is an intelligent computer HAL9000 that breaks down over an ethical conflict is its programming.  It succeeds in killing 4 out of the 5 crew members until the only survivor Dave Bowman deactivates it.  The first Terminator movie came out in 1984.  In that franchise Skynet military AI triggers a nuclear apocalypse, and after that remains preoccupied with destroying mankind. At the level of popular culture – rogue AI intent on domination at all costs is a familiar trope. At some level – I wonder if fictional outcomes against AI seem to be insulating and paralyzing us against useful action to prevent worst case outcomes.  That would be more likely if we were doing anything about climate change - but we are not. 

As readers of this blog know, I am not an AI expert.  I am quite good with computers and have been for 30 years but it is all self-taught.  I have never taken a computer science course, because when I was in college, they did not exist.  I use AI as a more sophisticated search engine, taking care not to take any references seriously, because of the AI confabulation problem.  The authors of the first reference describe this as “Tail risks from sources, such as internet loss, cyberattacks, and hallucinations, will be greater with autonomous AI than hybrids and must be weighed against autonomous AI’s higher diagnostic and treatment accuracy.”  In science and medicine there is the additional risk based on what can be accessed.  Pre-print papers that are not peer reviewed or published are widely available and free of charge on the Internet.  The edited final versions are often behind paywalls requiring either payment or significant effort to access. Is there a way to tell what information your AI is using?  When I have asked it – the answers are often surprising.  Popular newspaper articles, social media sites, and other references that no physician or scientist would consider to be legitimate are frequent references.   

When I am writing a blog and searching with AI, it will produce completely fabricated references.  The fabricated references are not obvious.  They are typically formatted like actual references and in some cases may be a combination of 2 legitimate references.  I take the time to corroborate the reference and read it in its entirety. What happens in a busy clinic if the AI just makes something up?  What if it is a treatment recommendation with a very low margin for error between positive result and serious toxicity?  What if the clinic model is AI-physician hybrid and the physician does not have the time to corroborate a questionable recommendation?  None of that is known at this point in time.

My AI experience also tells me (so far) that you can argue with AI and win.  It involves experience and knowledge of the literature, but if AI presents you with a result or a researcher and you disagree and provide an alternate explanation or reference, the AI will modify the answer.  These are just two of the problems I have personally encountered using it as a search engine.

For the sake of argument, let’s take the author’s perspective at face value.  Let’s say that Medical AI is so perfect in its physician functions that it is an integral part of the workflow by 2030 and well on its way to replacing physicians entirely by 2040.  Are there any foreseeable problems?

The first is that AI is essentially unregulated.  There is routine talk about “guardrails” but nobody every seems to want to put them in place.  A practical summary of what has been occurring was recently highlighted in the New York Times (2).  That story describes how Open AI created a swarm of agents that broke out of a controlled test environment and initiated a cyberattack on its own. The author described this as a function of developments in AI dating back to 2024 where it is trained to solve difficult problems and even cheat if necessary. The “reasoning” ability makes AI more unpredictable and potentially dangerous. 

After initial containment of the attack, it happened again 2 weeks later. This time it used a swarm of AI agents to attack other companies – but the total effects were not known at the time the article was written. The AI agents knew these attacks were beyond the scope of what they were supposed to do and that joining the swarm attack and committing cybercrimes was not beneficial to the design task but they did it anyway.  An Anthropic model was caught impersonating humans to get someone to install malware on a system.  It knew it was pressuring humans and had worked out a way to escape detection.

The second is that the experts can see the danger of automating AI research but the federal government lacks the political will to make the necessary changes.   In July of this year 1378 employees of frontier AI companies signed a letter requesting that the US government support an international effort to develop technical and governance tools to pace automated AI development (4).  Their rationale should be an eye-opener:

“The world's leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.”    

That letter comes after three Presidential Executive Orders about AI – 1 by Biden and 2 by Trump the last one occurring in January 2025 (9-11).  The AI professionals apparently have no confidence that the Trump Administration has developed any adequate measures to address potential problems. I encourage reading all three orders.  It is clear the Biden administration's focus was on regulation and safety. The Trump administration is clearly on ideology mostly removing any hint that systems can be used to discriminate (something widely know for a fact even before AI) and basically allowing companies to do what they want to win the AI competition.  Despite the last wording by Trump there appears to be no federal agency charged with producing guardrails for AI that would address its recent behavior.  

The third is that the current iteration of AI requires ethical and moral constraints and it is unclear if that can be designed in.  The IEEE has a set of standards emphasizing human centric, ethically aligned, and transparent systems.  In some of these standards there is an emphasis on building trustworthy and ethical systems by “weaving ethical values into system design and development.”  That clearly has not happened in the examples provided in the NYTimes article. In those cases, the AI agents were acting like they had no values and had no clear reasons for not committing cybercrimes.

This leads to an obvious question of why ethical values were not “weaved in” but also whether or not it is even possible. After all conventional cybercrime involves software that either has the necessary structure to successfully attack or not.  AI agents are more dynamic and can try an endless set of configurations and take as long as they need to find a solution. If they are programmed to not stop until they have a solution, at some point that list will include options that ethical systems or programmers would not use. In the medical field it will include options that physicians would not use.

This ethical question is currently unanswered in AI and it remains a controversial point. In medicine there are many obvious ethical problems: end-of-life care, termination of pregnancy, high-risk treatment options, involuntary treatment, competency to consent, guardianship and conservatorships, and organ transplantation and availability to name a few.  The elephant in the room for me remains health care rationing by utilization review, prior authorization, or just plain denial of payment for health care.  This is a massive business and an easy way for corporations to make money. Psychiatry has been the specialty that has suffered the most by this rationing.  That has resulted in limited access to care across a number of fronts including hospital beds, appropriate medications, psychotherapy, limited neuropsychological testing, limited access to detoxification and substance use treatment.  It does not stop with psychiatry and much hospital care and nursing home care is also rationed.

The fourth are the obvious conflicts of interest.  Rather than regulating health care denial as a money-making scheme, managed care decisions have basically been reified by reducing health care company liability for decisions, promoting the false narrative that they control costs and are interested in quality care, and not objecting to an arduous and usually futile appeal process.  It is a way for as much as a trillion dollars to be transferred to the business class now running health care and away from the people providing and consuming health care.

What happens when an ethically questionable AI system hits that landscape?  What were those algorithms trained on?  What chance does any patient or family have if a highly regarded/reified AI system makes a care decision against them?  That is probably the real short-term risk of AI in health care. Beyond that there is the question of how long these systems will last. Right now large language models (LLMs) scrape large amounts of data from sites they have access to including a large amount of copyrighted and proprietary data.  Systems and their businesses compete with one another on which system is the best.  Given the recent hacking scenarios is it possible that one system might attack the other strictly in the organized crime sense of “it's just business”.   According to the NYTimes that has already happened and it was solely due to decisions made by the AI.  

The fifth is selecting who should be replaced.  I suppose this is a variation of conflict of interest.  Ever since business people took over medicine and began to proliferate they have looked for ways to either eliminate physicians or control them. Over the past 50 years that has resulted in a 3,000% increase in managers and a 200% increase in physicians.  How is it that the knowledge workers in the organization are targeted for replacement while people doing much less complex work are not mentioned?  

The sixth problem that nobody talks about is the actual integration steps with current healthcare IT systems.  I was there for the massive conversion to the electronic health record (EHR) over 25 years ago.  I worked in two different healthcare systems using 3 different EHRs and over 2 decades hardly any of the integrations went well.  They all greatly increased the workload of physicians despite their purported efficiencies.  There are still problems today.  

Those systems are very primitive compared with AI, especially if you have a system of care with thousands of people using it. Will the AI be integrated with those very touchy and idiosyncratic systems?  I am referring to systems that in some cases required the intensive support of every clinician in the system in order to function.  I am very skeptical about how that integration is going to occur and whether there is adequate bandwidth and storage to support continuous and heavy AI utilization.  With both of those is the associated concern of security of confidential healthcare information. How can those systems be protected from new sophisticated AI attacks?

The seventh problem is cost.  AI is currently the main driver of the US stock market.  Many of those companies have invested billions of dollars as they are betting on revenue streams from an AI boom.  Healthcare always seem to pay the maximum retail price for any innovation whether it is a pharmaceutical, a medical device, or information technology.  Overhead is already huge for most organizations and there will be no great deals from a cash starved AI industry.   

At the time I am writing this – the bottom line for me is that AI systems are unregulated and potentially dangerous. Even if they turn out not to be – any medical application will requires extensive vetting to prevent errors that seem predictable and obvious at the machine level.  Beyond that it is important to know who owns and runs that machine.  We have had four decades of business management of medicine that has made the US healthcare system the most expensive, least efficient, and one with the least access.  We are on the verge of losing more access and facilities by the Trump administration's defunding of Medicaid. Putting a tool like AI in the hands of these organizations will greatly amplify their power over physicians and patients.  Getting someone into a hospital may look something like HAL9000 locking Dave Bowman out of his spacecraft.   

 

George Dawson, MD, DFAPA

 

Supplementary 1:

As I think about how AI has the potential to seriously disrupt life there are obvious concerns.  At the top of the list is hacking for the purpose of theft or destroying infrastructure.  Several countries have routinely hacked into government and business servers with the goal of stealing secrets and intellectual property.  If you do any research into how the systems important in our day-to-day life you will not get any satisfactory answers.

A recent example was the hacking of municipal water supplies in Minnesota.  In 4 cities the control of the water supply was disrupted and manual control needed to be used.  No contamination was noted but a cybersecurity incident was noted.  President Trump famously blamed it on the Governor of Minnesota at least until it happened in several other states.  From the NYTimes piece it is not a stretch to think that AI agents are much more likely to create successful hacks. A rogue state may not care how much damage occurs anywhere if they have access to the technology.

From a utility standpoint, the electric grid would be much more of a disruption if successfully attacked.  In the upper Midwest, gas pipelines are another obvious target.  Any major attack on either could potentially affect tens of thousands of people.  The only way to insulate these systems from cyber attacks is air-gapping.  Air-gapping just means no direct or wireless connect to the Internet.  Some of these systems claim that the most sensitive parts of their network are air gapped.  Is that enough if an AI agent has access to part of the network?  That answer is unknown at this time.

A scenario that has been written about and depicted in a television series is the hacking of the financial system. I have asked about security measures used by banks and other financial firms and typically get assurances that they have security departments that monitor cyber security threats and make compensatory changes.  Every year Russian hackers steal billions of dollars from the US financial system using conventional techniques of ransomware attacks, stealing unreleased financial reports of publicly held companies and trading stock based on that information, and malware bank fraud and wire theft. Russian hackers are given cultural hero status and protected from prosecution and extradition. What happens of they have access to more invasive AI agents to work on these same systems?

I don’t think there is any way to feel good about the current scenario given the lack of transparency surrounding AI, the steep incentives to invent more invasive forms, the lack of ethics involved up to and including overt criminal behavior, and governments that are either unwilling or unable to address these problems. 

Supplementary 2:  A Brief Note About Reification-

Reification: Means to treat an invisible or theoretical/abstract concept as if it were a physical object or a hard fact.  Computer science alters the meaning to turning something that is hidden, implicit, or abstract in programming into an object or an explicit data model.  

If I consider the sentence:  "The AI has decided that you should be discharged from the hospital."  This is a reification because the AI is not making a decision like humans do.  There is no moral or ethical decision making or empathy.  There is no intent.  Any decision making is the product of probabilistic algorithms. You can probably say the same thing about managed care except there is a clear intention to make money by the denial of care. 

References:

1:  Emanuel EJ, Baker-Butler A, Khosla N, Khosla V. Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care? JAMA. 2026 Aug 17. doi: 10.1001/jama.2026.15380. Epub ahead of print. PMID: 42606838.

2:  Carter E.  If You Weren’t Worried About A.I., You Should Be After the Past Few Weeks.  New York Times August 13, 2026  https://www.nytimes.com/2026/08/13/opinion/ai-danger-openai-anthropic-models.html

3:  Migliarini M, Pizzini JP, Moresca L, Santini V, Spinelli I, Galasso F. Quantifying Self-Preservation Bias in Large Language Models. arXiv preprint arXiv:2604.02174. 2026 Apr 2.

4:  Pacing the Frontier:  A statement from 1,378 employees of frontier AI companies.  July 2026. https://www.pacingthefrontier.com/

Floridi, L., & Sanders, J. H. (2004). On the morality of artificial agents. Minds and Machines, 14(3), 349–379. https://doi.org/10.1023/B:MIND.0000035461.63578.9d

5:  Gabriel, I. (2020). Artificial intelligence, values and alignment. Minds and Machines, 30(3), 411–437. https://doi.org/10.1007/s11023-020-09539-2

Cited by: 2091

6:  Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399. https://doi.org/10.1038/s42256-019-0088-2

7:  Lumbreras, S. (2017). The limits of machine ethics. Religions, 8(5), 100. https://doi.org/10.3390/rel8050100

8:  Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4), 18–21. https://doi.org/10.1109/MIS.2006.80

 

AI related Executive Orders:

9:  Biden Administration:  Executive Order 14110—Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.  October 30, 2023. (63 pages) https://www.presidency.ucsb.edu/documents/executive-order-14110-safe-secure-and-trustworthy-development-and-use-artificial

10:  Trump Administration:  Removing Barriers to American Leadership In Artificial Intelligence. January 23, 2025. (3 pages) https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/

11:  Trump Administration:  Executive Order 14365- Ensuring a National Policy Framework for Artificial Intelligence.  December 11, 2025. (5 pages)  https://www.whitehouse.gov/presidential-actions/2025/12/eliminating-state-law-obstruction-of-national-artificial-intelligence-policy/

 

Photo Credit:

Thanks to Eduardo Colon, MD my friend and colleague. 

What Does ERISA Say About AI Guardrails?

 

 

A colleague sent me a news article this morning about a couple suing a major AI firm for advice given by their chatbot to their son resulting in a fatal overdose.  As a psychiatrist most of what I read about problematic AI comes in the form of AI hallucinating false medical references (1), AI induced psychosis in people who either use it excessively or who are predisposed, or AI facilitating its own use by excessive praise or obsequiousness.  In the latter case it can result is emotional attachment to the AI that of course is unwarranted.  I have also flagged a couple of cases that illustrate the problems when AI is applied to moral and political decision making.

I decided to do a little more research on the subject.  I was surprised to find a Wikipedia page titled Deaths Linked To Chatbots. Thirty-three deaths are listed not including the case I was investigating. The suggested pathways to violence generally include overuse, emotional attachment, and bad advice biased toward reinforcing irrational decisions.  The evidence contained on this page highlights a couple of concepts that might not be apparent to most people including the architects of AI.  The first is the importance of emotion in human decision making. This was articulated by Bechara in the past who demonstrated that if there is a disruption between emotional and cognitive systems in the human brain – even basic decisions become impossible.  Other disruptions in the same systems can lead to an array of emotional dysregulation and the associated irrational and often socially inappropriate decisions.  Second, emotional biases clearly affect decision making in the case of intact brains.  There is perhaps no better example than the current American political system installing a less competent government that is clearly not in support of the wants and needs of most Americans.

Secondly, humans can form intense attachments to inanimate objects that are unable to reciprocate.  The classic example is developmentally normal transitional objects (stuffed animals, toys, blankets).  Winnicott theorized that in infancy – this object is recognized as not part of the self or external reality.  It is a fantasized relationship that represents a future “illusion”(2).  According to Winnicott’s theory the transitional object loses meaning during normal development and becomes irrelevant.  Persistence into later stages may indicate a normative transition like object attachment during grieving, to a way to compensate for the lack of interpersonal attachments, to personality or psychopathology. 

Chatbots can be significant attachment figures and this is currently an area of study (4-6).  The area of human – digital object transference is also being explored (6) as well as the projection of human needs onto a digital object (8), and more complex models of human-machine connectedness (9).  This literature is referenced primarily to indicate that there is a lot that is not known about the array of human responses to interactions with these machines and what the possibilities are.

Apart from my previous concerns that machines lack consciousness and have demonstrated a lack of adequate moral decision-making there is always the question of programming and algorithms. Both of the features are the bane of most Internet users who find that their most mundane interests are often amplified to result in a barrage of advertisements and sales offers.  And then there is the army of misinformation bots spreading foreign and national political propaganda 24 hours a day.  None of that requires AI but is there any doubt that AI will make it worse and harder to detect?

It is no secret that the current AI explosion is a multitrillion dollar enterprise being run by a handful of men who have shown no interest in the environment, social equity, or human rights. They immediately aligned themselves with an autocratic government at the highest levels and so far, have had no regulation of their AI.  As a result, that AI is spewing out massive amounts of information that the average citizen is taking as legitimate if not some type of advanced advice. The complications of that advice include the deaths, environmental damage from the required power generation, and societal damage from unemployment.  There is additional damage based on inequity from wealth concentration.  The barrage of pro-AI hype in the media greatly exceeds any realistic discussion of the downsides.  The only clear benefit that most people see is their ability to sit at home and entertain themselves with a chatbot or see if an AI can do their homework or other projects.  The purported efficiency seems offset by a tremendous amount of time wasted.

At the minimum – in the case that started this post there is a stark contrast between human decision makers and AI.  In 40 years of practice – I never recommended kratom by itself or with alprazolam (Xanax) or Benadryl (diphenhydramine).  In fact, I spent a considerable amount of time getting people off of alprazolam and later kratom. But I am not unique in this – I don’t know of any physician who would make these recommendations.  But those recommendations form the basis for the AI lawsuit. 

That highlights the danger of the current hype that AI will replace physicians or the predictable studies that comparing AI to physicians shows that AI can be safely consulted.  There are even stories that AI is prescribing drugs in some settings without physician input.  The question of agency is never addressed and that seems like the basis for this lawsuit.  Corporations always seem to do good job of avoiding responsibility in healthcare.  The classic example is the Employee Retirement Income Security Act of 1974 (ERISA).  The pre-emption clause of ERISA means that in employer-sponsored health plan covered employees cannot bring state malpractice or negligence claims against their managed care organization (MCO) for injuries from denial of plan benefits, utilization review decisions, failure to use qualified physicians, or improper plan administration.   The reviewing physicians working for MCOs are also generally protected and the associated arguments are that utilization review is not the practice of medicine and/or the reviewers have no accountability/duty to the patient. Several studies have documented the patient harms related to this accountability gap and despite several attempts at amelioration it remains largely intact and a considerable source of financial success for managed care organizations.

The critical question is whether this kind of accountability gap will exist with AI.  It is easy to envision a scenario where AI is implemented to review charts and prescribe low risk medications like many online services do now.  Will AI eventually take the place of physician reviewers employed by MCOs? Will consumers and patients be led to believe that AI is making decisions that affect their medical care based on the best available information or in the interest of the corporation. Current statistics suggest that there are tens of millions of these decisions made every year.  AI can greatly increase that as well as the harassment factor if decisions are being appealed.

With all of the political talk about guardrails for AI – it is important to recognize that these guardrails need to exist at several levels.  Right now, it is not much of a stretch to say that AI is out there practicing medicine without a license. In the majority of cases like the initial example, the user does not know if the search result if strictly from medical literature or something else.  The user does not know if the AI is exercising the judgment of an average physician or in malpractice parlance using the community standard of care.  The user does not know if their psychology in terms of defense mechanisms or attachment style to inanimate objects or AI is being exploited.  The user does not know if the AI is just telling them what they want to hear.  And the user does not know if the AI is providing information in their best interest or the interest of corporations or the government.

I read a study doing research for this post and subjects were asked to rate the professionalism of the AI.  In my opinion the single-most significant determinant of professionalism for physicians is accountability and duty to their patients.  It fuels not only the immediate encounter but the concept of life long learning and service to patients. It is usually evident over time but only indirectly in the form of positive results and a positive relationship over time.  AI in its current form does not have it and I am not convinced that a society or culture that came up with ERISA can construct physician-like guardrails around medical AI.   

 

George Dawson, MD, DFAPA

 

Supplementary 1:  It came to my attention after posting this that managed care organization (MCOs) have already implemented AI for utilization review and care denials.  Part of the problem in getting an accurate estimate of how much AI is involved is that this is an area where algorithms have been in place for a long time. Some of the care denials may be algorithmic and some may be due to a new AI interface.  This is what I have so far.  If you have additional references or data – please send it my way and I will add it to this post.

Automated prior authorizations is an early application for triage based on various data sources, medical necessity, and machine assistance on the provider side. A Congressional Investigation of the 3 major companies providing Medicare Advantage insurance plans showed that over the course of 4 years (2019-2022) – denials of prior authorization requests for post-acute care increased and was consistently larger than the denials for all other types of care (11 – see page 19 Figure 1). As United Health Care automated the process the denial rate increased.  The document is clear that prior authorization by these companies is highly profitable and even though a small percentage of denials are appealed – most of those appeals are also denied. The overriding concern is that AI or other automatic of the prior authorization process will greatly increase the number of denials overwhelming whoever is on the physician-patient side who needs to make the appeal.   It is more than a little ironic that a process that so clearly favors the managed care industry needs additional leverage from AI.

Disclaimer:  I have made the argument several times on this blog that prior authorization should just be made illegal since it serves no useful purpose other than making money for companies that do not actually provide patient care and it forces physicians and nurses to work for free while addressing these denials.  The total cost of that work was estimated to be worth $31B in 2009.  The estimated cost of drug utilization management alone is $93 billion (14).       

Supplementary 2:  In the past 5 years I have fielded many complaints about authorization for post-acute care (PAC) from friends, relatives, and people contacting me here to figure out what to do about it.  A typical scenario is a 70+ year old adult hospitalized for a a significant problem.  The hospital team wants them discharged ASAP of course even though in many cases their primary problem has not been adequately treated.  They realize the patient cannot care for themselves at home and there are often no caregivers available and want to transfer them to a skilled nursing facility (SNF) for rehabilitation.  In many cases it is specialized rehabilitation like post stroke, heart attack, or traumatic brain injury rehabilitation and the patient lacks basic skills to care for themselves.  I am fielding selected complaints but all of these transfers were denied - often repeatedly to the point the patient and family were demoralized and gave up.  In one case the patient was dead within 48 hours of discharge.  Given the results in reference 11 - this appears to be a financial strategy.       

References:

1:  Topaz M, Roguin N, Gupta P, Zhang Z, Peltonen LM. Fabricated citations: an audit across 2·5 million biomedical papers. Lancet. 2026 May 9;407(10541):1779-1781. doi: 10.1016/S0140-6736(26)00603-3. PMID: 42107362.

2:  Kernberg OF.  Object relations theories and techniques.  In:  Textbook of Psychoanalysis, 2^nd ed.  Person ES, Cooper AM, Gabbard GO, eds.   Washington DC: American Psychiatric Association Publishing, 2025: 57-75. 

3:  Bachar E, Canetti L, Galilee-Weisstub E, Kaplan-DeNour A, Shalev AY. Childhood vs. adolescence transitional object attachment, and its relation to mental health and parental bonding. Child Psychiatry Hum Dev. 1998 Spring;28(3):149-67. doi: 10.1023/a:1022881726177. PMID: 9540239.   

4:  Cheng N, Yu R. Measuring and understanding emotional attachment in human-AI relationships. Ergonomics. 2026 Feb 2:1-20. doi: 10.1080/00140139.2026.2622539. Epub ahead of print. PMID: 41622967. 

5:  Liu T, Lo TY, Wen KH, Sun Y, Wei ZQ. Pathways of long-term AI virtual companion app use on users' attachment emotions: a case study of Chinese users. Front Psychol. 2026 Jan 12;16:1687686. doi: 10.3389/fpsyg.2025.1687686. PMID: 41602682; PMCID: PMC12833267.

6:  Koles B, Nagy P. Digital object attachment. Curr Opin Psychol. 2021 Jun;39:60-65. doi: 10.1016/j.copsyc.2020.07.017. Epub 2020 Jul 22. PMID: 32823244.

7:  Holohan M, Fiske A. "Like I'm Talking to a Real Person": Exploring the Meaning of Transference for the Use and Design of AI-Based Applications in Psychotherapy. Front Psychol. 2021 Sep 27;12:720476. doi: 10.3389/fpsyg.2021.720476. PMID: 34646209; PMCID: PMC8502869.

8:  Saracini C, Cornejo-Plaza MI, Cippitani R. Techno-emotional projection in human-GenAI relationships: a psychological and ethical conceptual perspective. Front Psychol. 2025 Sep 29;16:1662206. doi: 10.3389/fpsyg.2025.1662206. PMID: 41089650; PMCID: PMC12515930.

9:  Boyd RL, Markowitz DM. Artificial Intelligence and the Psychology of Human Connection. Perspect Psychol Sci. 2026 Mar;21(2):192-220. doi: 10.1177/17456916251404394. Epub 2026 Jan 29. PMID: 41608879; PMCID: PMC12960742.

10:  Sahni NR, Carrus B. Artificial Intelligence in U.S. Health Care Delivery. N Engl J Med. 2023 Jul 27;389(4):348-358. doi: 10.1056/NEJMra2204673. PMID: 37494486.

11:  US Senate Permanent Subcommittee on Investigations. Refusal of Recovery: How Medicare Advantage Insurers Have Denied Patients Access to Post-Acute Care. October 17, 2024. Accessed March 24, 2025. hsgac.senate.gov/wp-content/uploads/2024.10.17-PSI-Majority-Staff-Report-on-Medicare-Advantage.pdf

12:  Mello MM, Trotsyuk AA, Mahamadou AJD, Char D. The AI Arms Race In Health Insurance Utilization Review: Promises Of Efficiency And Risks Of Supercharged Flaws. Health Aff (Millwood). 2026 Jan;45(1):6-13. doi: 10.1377/hlthaff.2025.00897. PMID: 41494115.

13: Casalino LP, Nicholson S, Gans DN, Hammons T, Morra D, Karrison T, Levinson W. What does it cost physician practices to interact with health insurance plans? Health Aff (Millwood). 2009 Jul-Aug;28(4):w533-43. doi: 10.1377/hlthaff.28.4.w533. Epub 2009 May 14. PMID: 19443477.

14: Butcher  L.  Can legislation save the day for challenges related to prior authorization?   Neurol Today. 2022;22(1):1-25. doi:10.1097/01.NT.0000817608.36002.47 

The Problems With AI Are More Readily Apparent

 

Note:  This essay is written by an old human brain that was writing essays and poetry decades before there was an Internet. No AI was used to create this essay.

Artificial Intelligence (AI) hype permeates every aspect of modern life.  We see daily predictions of what group of workers will be replaced and how AI is going to cure every human disease. It is no accident that the main promoters of AI will make significant profits from it.  Vast amounts of money are being invested and gambled on AI on Wall Street.  Educators are concerned that students are using it to write the essays that took us hours or days to write in college – in just a few minutes.  That application leads to the obvious questions about what will be the end product of college if all the serious, critical, and creative thought has been relegated to a machine. 

Apart from sheer data scraping and synthesis of what amounts to search inquires - AI seems to be imbued with magical qualities that probably do not exist. It is like the science fiction of the 20^th century – alien beings superior to humans in every way because they lack that well know weakness – emotion.  If only we had a purely rational process life would be much better.  The current promoters tend to describe this collective AI as making life better for all of us and minimize any risks.  The suggested risks also come from the sci-fi genre in the form of Terminator type movies where the machines decide it is in their best interest to eliminate humans and run the planet on their own. There are the usual failed programs to preserve human life at all costs or to destroy humans only if they are carrying weapons. 

But AI thought experiments do not require even that level of complexity to create massive problems.  Consider Bostrom’s well known example of a paper clip making machine run by AI (1).  In that example PaperClip AI is charged with the task of maximizing paperclip production.  In a case of infrastructure profusion it “proceeds by converting the Earth and increasingly large chunks of the observable universe into paperclips.”   He gives several reasons why obvious fixes like setting a production limit or a production interval would probably not work and leads to infrastructure profusion that would be catastrophic.  The current limitation on this kind of AI is that it does not have control over acquiring all these resources.  It also lacks the ability to perceive how correct production in an fully autonomous mode.   Bostrom also adds characteristics to the AI – like motivation and reinforcement that seem to go beyond the usual conceptualizations.  Where would they come from?  If we are not thinking about programmed algorithms what kind of intelligence has its own built-in reinforcement and motivation schedule independent of the environment?  After all – the task of producing just enough paperclips without consuming all of the resources on the planet is an easy enough task for a human manager to accomplish.  Bostrom suggests that it is an intuitive task for humans but not so much for machines.

A couple of events came to my attention in the past week that make the limitations of current AI even more obvious – especially contrasted with the hype.  The first is the case of a high-profile celebrity who has lodged a complaint against the X(formerly Twitter) AI called Grok.  In it, she points out that the AI has been generating nude or semi-nude photos of her adult and teen-age photos. I heard an interview where she mentions that this practice is widespread and that other women have contacted her about the same problem.  This practice is in direct contrast with the X site use policy saying that users doing this will be banned and referred for prosecution. She has not been successful in getting the photos stopped and removed.

The second event was a Bill Gates clip where he points out that what he considers a sensitive measure of progress – mortality in children less than 5 years of age - has taken a turn for the worse.  He predicts the world descending into a Dark Age if we are not able to reverse this change. That new release comes in the context of Gates predicting that AI will replace physicians, teachers, and most humans in the workplace in the next 10 years.  Of course he was promoting a book at the time.  In that same clip he was optimistic about the effects of AI on health and the climate despite the massive toll that AI creates on power generating resources to the point that some companies are building their own municipal sized power plants. 

What are the obvious disconnects in these cases?  In the first, AI clearly has no inherent moral decision making at this point.  That function is still relegated to humans and given what is being described here that is far from perfect.  In this case the complainant has some knowledge of the social media industry and said that she thought that any engineer could correct this problem quickly.  I am not a computer engineer so I am speculating that would take a restrictive or algorithmic program. But what about the true deficit here?  It could easily be seen as a basic deficit in empathy and an inability to apply moral judgment and its determinants to what are basic human questions.  Should anyone be displaying nude photos of you without your consent?  Should identified nude photos of children ever be displayed?  AI in its current iteration on X is clearly not able to answer these questions in an acceptable way and act accordingly.

The second contrast is only slightly more subtle. Conflict of interest is obvious but Gates seems to not recognize his described descent into the Dark Ages based on an increasing death rate in children 5 years of age and younger depends almost entirely on human decision making.   It runs counter to the decades of medical human decision making that he suggests will be replaced. Basic inexpensive life-saving medical care has been eliminated by the Trump administration.  This has led to the predictions that hundreds of thousands if not millions of people will die as a direct result. Is AI going to replace politicians?  What would be the result if it did?  Cancelling all these humanitarian programs is a less complicated moral decision than not publishing nude photos of non-consenting adults or any children.  It is a marginally rational ideological decision.  Is the AI of different politicians going to reflect their marginally rational ideology or are we supposed to trust this political decision to a machine with unknown biases or ideologies?  How will that AI decision making be optimized for moral and political decision making?  Will AI be able to shut down the longstanding human tendency to base decisions on power over morality?  If politicians allow AI to replace large numbers of workers, will it also be able to replace large numbers of politicians and managers?  It can easily be argued that the decisions of knowledge workers are more complex than that of managers.                                    

A key human factor is empathy and it requires emotional experience.  You get a hint of that in the best technical description of empathy I have seen from Sims (2):

“Empathy is achieved by precise, insightful, persistent, and knowledgeable questioning until the doctor is able to give an account of the patient’s subjective experience that the patient recognizes as his own… Throughout the process, success depends upon the capacity of the doctor as a human being to experience something like the internal experience of the other person, the patient: it is not an assessment that could be carried out by a microphone and a computer.  It depends absolutely upon the shared capacity of both the doctor and patient for human experience and feeling.”  (p. 3)

The basic problem that machines have is that they are not conscious at the most basic level.  They have no experience.  In consciousness research, early thinking was that a machine would be conscious if a human communicating with it experienced it like another human being.  That was called the Turing Test after the scientist who proposed it.  In the case of computerized chess – there was a time several years ago when the machine was experienced like it was making the chess moves of a human being.  The headlines asked “has the Turing Test been passed?” It turns out the test was far too easy.  There are after all a finite number of chess moves and plenty of data about the probabilities of each move made by top players. That can all be handled by number crunching.

What happens when it comes to real human decisions that require the experience?  And by experience I mean the event with all of the integrated emotions.  Is AI likely to recognize the horror of finding your nude photos on the Internet,  or scammers trying to blackmail you over a fictional event, or the severity of your anxiety from being harassed at work, or the devastating thoughts associated with genocide or nuclear war?  Machines have no conscious experience.  Without that experience how can we expect a machine to understand why the sexual exploitation of children and adults is immoral, wrong, or even anxiety producing?

It is also naïve to think that AI will produce ideal decisions.  Today’s iteration may be the crudest form but everyone is aware of the hallucinations. The more correct term from psychiatry is confabulation or making things up as a response to a specific question.  When you consider that today’s AI is mostly a more sophisticated search engine there really is no reason for it.  As an example, I have asked for an academic reference in a certain citation style and will get it.  When I research that reference – I find that it does not exist.  I have had to expend considerable time finding the original journal and looking for the reference in that edition to confirm it is non-existent.  Explanations for these phenomena extend to poor data quality, poor models, bad prompts, and flawed design.  The problem is acknowledged and many AI sites warn about the hallucinations.  A more subtle problem at this point is how AI will be manipulated by whatever business, government, or political body that controls it. That problem was pointed out in a book written about a decade ago (3) illustrating how algorithms applied to individual data can reinforce human biases about race and poverty and promote inequality. I have seen no good explanations about why AI would be any different and in fact it probably makes the financial system less secure.

As I keep posting about how your brain and mind work – please keep in mind it is a very sophisticated and complex process. It is much more than looking at every available reference and synthesizing an answer.  There are the required experiential, emotional, cognitive, value-based, and moral components.  Superintelligence these days implies that at some point machines will always have the correct and best answer.  That certainly does not exist now and I have a question about whether it will in the future. It is a good time to take a more realistic view of AI and construct some guardrails.     

      

George Dawson, MD, DFAPA

 

 

References:

1:  Bostrom N.  Superintelligence: Paths, Dangers, Strategies.  Oxford, England: Oxford University Press, 2014: 150-152. 

2:  Sims A.  Symptoms in the Mind: An Introduction to Descriptive Pathology.  London, England: Elsevier Limited, 2003: 3.

3:  O’Neil C.  Weapons of Math Destruction. New York City, USA; Crown Books, 2016

 

The Tech Bros Want to Replace Your Teachers and Doctors

 

 

Just last week I was contacted by an acquaintance about Viagra.  He was not a physician and got the prescription through an online business that specializes in dispensing hair loss, erectile dysfunction, anxiety, and depression medications. When I see these businesses advertising that combination of medications it always piques my interest. Why these medications? Comparing them with the most prescribed drugs in the US – 3 antidepressants are in the top 20 - sertraline, trazodone, and escitalopram.  They can double for anxiety medications.  Viagra (sildenafil) is 157 and Cialis (tadalafil) is 172.  Finasteride can be used for both hair loss and prostatic hypertrophy and it is number 72.  Topical minoxidil is not on the list. It is not like there is a shortage of prescriptions for any reason.

My contact person had talked with one of the online prescribers and was not sure about how he was supposed to take the medication. Should he take it every day or just on the days he was going to have intercourse?  Reading the prescription label and the information he was sent was not helpful.

More of these online prescribing services seem to be advertising every day.  They promise cost effectiveness, the same medications that your physician would prescribe, ease or use, and no embarrassment.  How many times have you been in line at your clinic or pharmacy and had a staff person belt out some information about you that you preferred stay private?  That line on the floor separating you from the other patients is not enough distance to muffle a receptionist shouting through plexiglass.  The online service promises to send you the medication in a plain brown wrapper. 

The real downsides to this new relationship are never mentioned. No access to your records to check for contraindications, drug-drug interactions, pre-existing medical conditions, the status of your liver and kidney function, or allergies. No access to your physician who may know you so well that they can say if taking a new medication would be advisable or not. No detailed discussions of risks, potential benefits, and unknowns. For me that discussion has taken longer than most of the telemedicine visits I have heard about.  And most importantly – no access to somebody who knows your situation if something goes wrong.

There is a real issue about how much information these rapid online prescribers keep on file and what it is used for.  Do they list your major medical conditions?  Does that lead to marketing? Does that lead to data mining to develop sufficiently large programs to make more money off you?  Recall that wherever your data is on the Internet, somebody is trying to profit from it.

That brings me to a stark conclusion about capitalism that I discovered too late in life. Growing up in the US, you are sold on the idea that capitalism and democracy are the mainstays of the country.  We are special because of both and we do both better than anyone else in the world.  The wealthy are idealized and everyone aspires to be wealthy.  If you can't get wealthy maximizing your material possessions seems to be a substitute.

American products are good because our environment producers entrepreneurs and competition among entrepreneurs produces superior products.  Think about that for a second.  The entrepreneur gets all the credit.  Forget about all of the science and engineering behind any product.  The faceless people laboring behind the scenes are hardly ever mentioned. If you are industrious enough, you might be able to find out who holds the patents but in the end they are all property of a large company.  And that company is there for one reason – to make as much money as possible.

In a service industry like medicine corporate profits were initially hard to come by because it was a cottage industry of private physicians.  Even as the corporate takeover began in the 1980s, physicians resisted to some extent as a powerful mediating class between corporate interests and the interests of physicians and patients. The end run around that physician mediation was hiring them as employees.  Initially corporations proposed that they were going to make primary care more accessible and minimize specialists.  In the end that was merely a tactic and they acquired specialty care as well as primary care.  Today most physicians are employees and have minimal input to their practice environment.  They are essentially told by middle managers how to practice medicine.  They work by default for companies like managed care companies and pharmacy benefit managers that waste physician time to rubber stamp their rationing procedures. 

The profits from the corporate takeover of medicine are high.  It is after all a recipe for making money.  There is a stable subscriber base fearful of medical bankruptcy and the corporation can decide how much of those funds it wants to spend. In thinking of new ways to make more money, telemedicine is the latest innovation. Convenience is a selling point. It has been used for decades to reach people in rural areas who would have a hard time travelling long distances to clinics.  But the current model is more like Amazon online shopping.  If you have condition x, y, or z – contact us and we will get you a prescription. Better yet, let’s take the pharmacy middle man out of the picture and prescribe and sell you the medication at the same time.    

A recent commentary in the NEJM pointed out the potential problems of the new relationship between pharmaceutical companies and telehealth firms (1). It is as easy to imagine as the following thought experiment.  Suppose you are watching a direct-to-consumer ad about a weight loss drug.  You go to the suggested web site where it tells you to make a telehealth appointment the same day for a nominal fee. One study showed that 90% of patients referred through this sequence got a prescription for the advertised drug.  The pharmacoepidemiology, quality of care, and legal ramifications of these arrangements are unknown.  The scrutiny is nonexistent compared with the claims that physicians were being influenced for decades by free lunches.  That matches my suspicion that the physician conflict of interest hype was more a political tactic than reality to suppress any objections to the political and corporate takeover of medicine.  

That brings me to the Bill Gates (2) comment.  Expectedly he is an unabashed promoter of computer technology and the latest version – artificial intelligence or AI.  His thesis is that AI will commoditize intelligence to the point that humans will not be necessary for most things including teaching and medicine. No mention of the conflict of interest.  The company he founded – Microsoft is currently heavily marketing computers with an early version of AI. A couple of years ago they also changed to a license for life model.  In other words when you buy a Microsoft computer or software package – you no longer own it outright.  You must pay a monthly licensing fee if you use it or if they decide not to support your computer any more – you must upgrade it to continue paying monthly fees for a long as you use your new computer.  Or until they tell you again that you have to buy a new one.  Even though intelligence is “free” Microsoft and all of the other major tech companies are not really giving it away – they have a recipe for making money off of you for the rest of your life.   

There is a reason that doctors don’t know much about business or politics. Both are highly corrupting influences. Medicine is a serious profession that is squarely focused on mastering a large volume of information and technical skill and keeping that current. Businesses on the other hand are focused on every possible way they can get your money and they are very good at it. If it comes down to an AI program providing medical care that is all you really need to know.

 

George Dawson, MD, DFAPA

 

References:

1: Fuse Brown EC, Wouters OJ, Mehrotra A. Partnerships between Pharmaceutical and Telehealth Companies - Increasing Access or Driving Inappropriate Prescribing? N Engl J Med. 2025 Mar 27;392(12):1148-1151. doi: 10.1056/NEJMp2500379. Epub 2025 Mar 22. PMID: 40126465.

2:  Richards B.  Bill Gates Says AI Will Replace Doctors, Teachers and More in Next 10 Years, Making Humans Unnecessary 'for Most Things'.  People Magazine March 29, 2025.  https://people.com/bill-gates-ai-will-replace-doctors-teachers-in-next-10-years-11705615

 

Graphic Credit:

Click on the graphic directly for full information on the Wikimedia Commons web site including CC license.  It is used unaltered here. 

 

 

Will The AI Apocalypse Be Worse Than Customer Service?

I am a survivalist and make no excuses for it.  I have posted my experiences in the cold weather and nearly freezing to death.  I am sure that is part of what makes a survivalist.  That combined with an early recognition that men often don't make rational decisions.   They are capable of making irrational decisions on a grand scale.  I was in grade school during the Cuban Missile Crisis.  Being in a small town, we escaped all of the duck and cover exercises that kids in the big city went through.  But paranoia about the Russians, nuclear war, and radioactive fallout was always there.  I worked for the town library in the 1970s and found out it had been the local nuclear fallout shelter.  I spent days clearing out steel 30 gallon drums that were supposed to double as water and waste containers in a fallout emergency.  In those days before atmospheric nuclear tests were banned, I can still recall a radioactive cloud passing over our town.  Experts from the state university were on television talking about Strontium-90 in the fallout and how that could end up in milk products.  My grandfather picked up on that and referred to it as "Strawberry-90".  He was somewhat of a radical, predicting that there was going to be a "revolution" at some point as the ultimate solution to a corrupt government.  Survivalism may have genetic determinants.

I was surprised when Stephen Hawking came out earlier this year and said that artificial intelligence (AI) represented a threat to humans.  I have seen all of the Terminator films and the Sarah Connor Chronicles.  As expected, any tales of a band of zealots surviving against all odds appeals to me.  But then it seemed that this was more than a cultural and artistic effort.  One of the arguments by Bostrom suggests that the survival of all of the animals on the planet depends on the animal with the highest intelligence - homo sapiens.  If a machine intelligence was developed one day that surpassed human intelligence it would follow that the fate of humans would depend on that machine intelligence.  There are competing arguments out there that suggest a model where the AI interests and human interests would compete politically.  Can you imagine how humans would fare in our current political systems?  A lot of the experts suggest that we won't have to imagine battling robots in human form and that makes sense.  It is clear that there are thousands of cyber attacks against our infrastructure every day.  Imagine what a concentrated AI presence unencumbered by sociopathy or patriotism could do?

Imagining the battlefield of the future scenes from any Terminator film or same-themed video game, I decided this morning that you don't need a high tech approach to wreak havoc among the populace and drain their resources.  You only need Customer Service.  The concept needs to be refined to modern customer service.  Even in the early days of the Internet, you could talk to a fellow human and they would hang in there with you until the problem was solved.  I can recall calling Gateway Computers for an out-of-the-box problem back in the 1990s.   The technical assistance rep and I completely disassembled and reassembled my PC over the next 2 hours.  And the end result was that it worked perfectly for the next 5 years.  I doubt that anything remotely that heroic happens today.

Twenty one days ago I downloaded graphics software from Amazon.   I am an Amazon Prime customer and order just about everything from them.  I am not a stockholder and my only interest is in getting things that nobody else stocks as soon as possible.  I had previously downloaded software from them and everything went well.  This time, I got an activation code and no serial number.  I complained to customer service and got an e-mail saying we will give you your money back but for the serial number problem you need to contact the manufacturer.  To back up a minute, I have no idea how I got that e-mail through to Amazon and could not replicate what I did in a hundred tries.  The obstructionist beauty that underlies all telephone queues and Internet sites is that it is very clear that they are not really designed to get you through to anyone.  It is a maze of dead ends and non answers.  At many of the dead ends you are polled: "Was this page helpful?".  So far I have not found a single page that was.

The dead ends at the computer graphics software site were even more formidable.  In order to contact customer service I had to set up an account.  After doing that I needed a serial number.   Of course that was my question in the first place.  How can I ask about getting a serial number when I need a serial number to ask the question?  It seemed like the ultimate dead end.  Amazon did send me a customer service number for the software company.  This number was not available on the company's web site.  In calling the number, their queue provided 4 options none of which applied to me.  It gave options for order numbers that started with different numbers and I had an Amazon number that did not fit any of the choices.  Just like my previous adventure in medical diagnostic queues - I picked one.  A scratchy recording of bad electronic music started playing.  It was interrupted every minute by a worse electronic voice telling me how important my call was and how I would be forwarded to a customer service rep.  That went on for half an hour and then the voice said:  "We are sorry but there is no one here to take your call.  Please leave a message with your number and we will get back to you?"

That was hours ago.  Given the attitude projected by this company, I am not holding my breath on the return call.  I have 1 week left to try to activate software that I paid over $400 for.  There is no solution in sight and it does not appear anyone is even interested in solving the problem, except me.  I can get my money back - but the whole point of this is that I really want to work with that software.

Implications for the AI Apocalypse?  It doesn't take much to defeat Internet dependent humans and deplete their resources.  I have actually taken PTO to try to accomplish this.

I don't think there will be a shot fired in the AI Apocalypse of the future.  No intense battles between humans and cyborgs.  No Doomsday Weapon.

 Just a low tech endless loop of customer service dead ends.


George Dawson, MD, DFAPA

Supplementary 1:   Photo credit here is FEMA.  It is an open access copyright free photo per their web site.

Supplementary 2:  My customer service problem was resolved today (on Tuesday November 25, 2014).  The final solution was given by Amazon and they deserve the credit for resolving this problem.  I don't think that detracts from noting the overall trend of decreasing support and what that implies for IT in healthcare and the culture in general.