AI never leaves the headlines. At least two AI CEOs have clearly stated that all human diseases will be wiped out in the next decade. There is some suggestion that they do not have a very good understanding of human disease since one referred to cancer as a single disease. Other people reacting to those headlines have suggested that will not happen. I think it is highly unlikely.
The medical literature is becoming less controversial. A recent opinion piece in JAMA suggests that
AI surpasses not only physicians but also AI assisted physicians in at least 5
cognitive tasks of physicians ranging from data acquisition to prescribed
therapies. The authors suggest that by
2030, AI alone will consistently produce better diagnostic and therapeutic
results than physicians. They do qualify
that opinion in several ways. First, the
test comparisons are generally not clinical examples but more like test
problems. Second, they point out that AI
alone cannot exceed physicians performance in radiology. That finding is quite surprising because it
suggests the pattern recognition aspects of human cognition alone are unique. Third, the literature is affected by
publication bias – negative AI studies are apparently not published. And fourth, there are barriers to
implementation but apparently not enough to keep the authors from concluding:
“Nonetheless, superior autonomous AI will likely be ready
to be deployed for real-world cognitive medical tasks in some, maybe many,
workflows by 2030.”
Let me digress for a paragraph about science fiction and the
pitfalls of AI. Science fiction writers have been warning about this for
decades. Kubrick’s classic 2001 A
Space Odessey came out in 1968. One of
the central features in the film is an intelligent computer HAL9000 that breaks
down over an ethical conflict is its programming. It succeeds in killing 4 out of the 5 crew members
until the only survivor Dave Bowman deactivates it. The first Terminator movie came out in
1984. In that franchise Skynet military
AI that triggers a nuclear apocalypse, and after that remains preoccupied with
destroying mankind. At the level of popular culture – rogue AI intent on domination
at all costs is a familiar trope. At some level – I wonder if fictional outcomes
against AI seem to be insulating and paralyzing us against useful action to
prevent worst case outcomes.
As readers of this blog know, I am not an AI expert. I am quite good with computers and have been
for 30 years but it is all self-taught.
I have never taken a computer science course, because when I was in college,
they did not exist. I use AI as a more
sophisticated search engine, taking care not to take any references seriously,
because of the AI confabulation problem.
The authors of the first reference describe this as “Tail risks from
sources, such as internet loss, cyberattacks, and hallucinations, will be
greater with autonomous AI than hybrids and must be weighed against autonomous
AI’s higher diagnostic and treatment accuracy.” In science and medicine there is the
additional risk based on what can be accessed.
Pre-print papers that are not peer reviewed or published are widely
available and free of charge on the Internet.
The edited final versions are often behind paywalls requiring either
payment or significant effort to access. Is there a way to tell what
information your AI is using? When I have
asked it – the answers are often surprising.
Popular newspaper articles, social medica sites, and other references
that no physician or scientist would consider to be legitimate.
When I am writing a blog, searching with AI, and it produces
completely fabricated references. I take
the time to corroborate the reference and read it in its entirety. What happens
in a busy clinic if the AI just makes something up? What if it is a treatment recommendation with
a very low margin for error between positive result and serious toxicity? What if the clinic model is AI-physician
hybrid and the physician does not have the time to corroborate a questionable
recommendation? None of that is known at
this point in time.
My AI experience also tells me so far that you can argue
with AI and win. It involves experience
and knowledge of the literature, but if AI presents you with a result or a
researcher and you disagree and provide an alternate explanation or reference,
the AI will modify the answer. These are
just two of the problems I have personally encountered using it as a search
engine.
For the sake of argument, let’s take the author’s
perspective at face value. Let’s say
that Medical AI is so perfect in its physician functions that it is an integral
part of the workflow by 2030 and well on its way to replacing physicians
entirely by 2040. Are there any
foreseeable problems?
The first is that AI is essentially unregulated. There is routine talk about “guardrails” but
nobody every seem to want to put them in place. A practical summary of what has been occurring
was recently highlighted in the New York Times (2). That story describes how Open AI created a
swarm of agents that broke out of a controlled test environment and initiated a
cyberattack on its own. The author described this as a function of developments
in AI dating back to 2024 where it is trained to solve difficult problems and
even cheat if necessary. The “reasoning” ability makes AI more unpredictable
and potentially dangerous.
After initial containment of the attack, it happened again 2
weeks later. This time it used a swarm of AI agents to attack other companies –
but the total effects were not known at the time the article was written. The
AI agents knew these attacks were beyond the scope of what they were supposed
to do and that joining the swarm attack and committing cybercrimes was not
beneficial to the design task but they did it anyway. An Anthropic model was caught impersonating
humans to get someone to install malware on a system. It knew it was pressuring humans and had
worked out a way to escape detection.
In July of this year 1378 employees of frontier AI companies
signed a letter requesting that the US government support an international
effort to develop technical and governance tools to pace automated AI
development (4). Their rationale should
be an eye-opener:
“The world's leading AI companies believe they could be
close to automating AI research. It is hard to predict exactly how much this
will accelerate AI progress, but there is a real risk that capability
development rapidly accelerates beyond our ability to understand or control the
resulting systems.”
That letter comes after three Presidential Executive Orders –
1 by Biden and 2 by Trump the last one occurring in January 2025. The AI professionals apparently have no
confidence that the Trump Administration has developed any adequate measures to
address potential problems. I encourage reading all three orders. It is clear the Biden administrations focus
was on regulation. The Trump administrations is clearly on ideology mostly
removing any hint that systems can be used to discriminate (something widely
know for a fact even before AI) and basically allowing companies to do what
they want. Despite the last wording by
Trump there appears to be no federal agency charged with producing guardrails
for AI that would address its recent behavior.
The IEEE
has a set of standards emphasizing human centric, ethically aligned, and
transparent systems. In some of these
standards there is an emphasis on building trustworthy and ethical systems by “weaving
ethical values into system design and development.” That clearly has not happened in the examples
provided in the NYTimes article. In those cases, the AI agents were acting like
they had no values and had no clear reasons for not committing cybercrimes.
This leads to an obvious question of why ethical values were
not “weaved in” but also whether or not it is even possible. After all
conventional cybercrime involves software that either has the necessary
structure to successfully attack or not.
AI agents are more dynamic and can try an endless set of configurations
and take as long as they need to find a solution. If they are programmed to not
stop until they have a solution, at some point that list will include options
that ethical systems or programmers would not use. In the medical field it will
include options that physicians would not use.
This ethical question is currently unanswered in AI and it
remains a controversial point. In medicine there are many obvious ethical
problems: end-of-life care, termination of pregnancy, high-risk treatment
options, involuntary treatment, competency to consent, guardianship and conservatorships,
and organ transplantation and availability to name a few. The elephant in the room for me remains
health care rationing by utilization review, prior authorization, or just plain
denial of payment for health care. This
is a massive business and an easy way for corporations to make money.
Psychiatry has been the specialty that has suffered the most by this
rationing. That has resulted in limited
access to care across a number of fronts including hospital beds, appropriate
medications, psychotherapy, limited neuropsychological testing, limited access
to detoxification and substance use treatment.
It does not stop with psychiatry and much hospital care and nursing home
care is also rationed.
Rather than regulating health care denial as a money-making
scheme, managed care decisions have basically been reified by reducing health
care company liability for decisions and not objecting to an arduous and often
futile appeal process. They have been
rationalized as a way to contain costs even though there is no evidence that
has occurred. It is a way for as much as
a trillion dollars to be transferred to the business class now running health
care and away for the people providing and consuming care.
What happens when an ethically questionable AI system hits
that landscape? What were those
algorithms trained on? What chance does
any patient or family have if a highly regarded AI system makes a care decision
against them? That is probably the real short-term
risk of AI in health care. Beyond that there is the question of how long these systems
will last. Right now large language models (LLMs) scrape large amounts of data
from sites they have access to including a large amount of copyrighted and
proprietary data. Systems and their
businesses compete with one another on which system is the best. Given the recent hacking scenarios is it
possible that one system might attack the other strictly in the organized crime
sense of “its just business”. According
to the NYTimes that has already happened and it was solely due to decisions
made by the AI.
At the time I am writing this – the bottom line for me is
that AI systems are unregulated and potentially dangerous. Even if they turn
out not to be – any medical application will requires extensive vetting to
prevent errors that seem predictable and obvious at the machine level. Beyond that it is important to know who owns
and runs that machine. We have had four
decades of business management of medicine that has made the US healthcare
system the most expensive, least efficient, and one with the least access. We are on the verge of losing more access and
facilities by the Trump administrations defunding of Medicaid. Putting a tool
like AI in the hand of these organizations will greatly amplify their power
over physicians and patients. Getting
some one in to a hospital may look something like this.
George Dawson, MD, DFAPA
References:
1: Emanuel EJ,
Baker-Butler A, Khosla N, Khosla V. Will Autonomous AI Exceed AI-Aided
Physicians as the Best Medical Care? JAMA. 2026 Aug 17. doi:
10.1001/jama.2026.15380. Epub ahead of print. PMID: 42606838.
2: Carter E. If You Weren’t Worried About A.I., You Should
Be After the Past Few Weeks. New York
Times August 13, 2026 https://www.nytimes.com/2026/08/13/opinion/ai-danger-openai-anthropic-models.html
3: Migliarini M, Pizzini JP, Moresca L, Santini V, Spinelli I, Galasso F. Quantifying Self-Preservation Bias in Large Language Models. arXiv preprint arXiv:2604.02174. 2026 Apr 2.
4: Pacing the
Frontier: A statement from 1,378
employees of frontier AI companies. July
2026. https://www.pacingthefrontier.com/
Floridi, L., & Sanders, J. H. (2004). On the morality of
artificial agents. Minds and Machines, 14(3), 349–379.
https://doi.org/10.1023/B:MIND.0000035461.63578.9d
5: Gabriel, I.
(2020). Artificial intelligence, values and alignment. Minds and Machines,
30(3), 411–437. https://doi.org/10.1007/s11023-020-09539-2
Cited by: 2091
6: Jobin, A., Ienca,
M., & Vayena, E. (2019). The global landscape of AI ethics guidelines.
Nature Machine Intelligence, 1(9), 389–399.
https://doi.org/10.1038/s42256-019-0088-2
7: Lumbreras, S.
(2017). The limits of machine ethics. Religions, 8(5), 100.
https://doi.org/10.3390/rel8050100
8: Moor, J. H.
(2006). The nature, importance, and difficulty of machine ethics. IEEE
Intelligent Systems, 21(4), 18–21. https://doi.org/10.1109/MIS.2006.80
AI related Executive Orders:
Biden Administration: Executive Order 14110—Safe, Secure, and
Trustworthy Development and Use of Artificial Intelligence. October 30, 2023. (63 pages) https://www.presidency.ucsb.edu/documents/executive-order-14110-safe-secure-and-trustworthy-development-and-use-artificial
Trump Administration: Removing Barriers to American Leadership In Artificial
Intelligence. January 23, 2025. (3 pages) https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/
Trump Administration:
Executive Order 14365- Ensuring a National Policy Framework for Artificial
Intelligence. December 11, 2025. (5
pages) https://www.whitehouse.gov/presidential-actions/2025/12/eliminating-state-law-obstruction-of-national-artificial-intelligence-policy/
