Healthcare AI hallucinates medical data up to 75% of the time, low frequency events most affected

Artificial intelligence is rapidly entering clinical healthcare settings, bringing both transformative potential and significant risks that medical professionals must navigate carefully. Two leading physicians examine how AI integration with electronic medical records could revolutionize patient care, while warning of critical challenges including AI “hallucinations” that occur up to 75% of the time.

What you should know: AI demonstrates remarkable diagnostic capabilities that can match or exceed experienced specialists in a fraction of the time.

• A recent study analyzing over 3 million emergency room visits found AI could predict patient agitation and violence, confirming the clinical principle that “past behavior is the best predictor of future behavior.”
• AI video analysis can detect subtle behaviors in premature infants that help clinicians optimize feeding and care timing—capabilities that human observers often miss.
• The technology excels at processing enormous datasets to inform clinical decision-making and patient management.

The big picture: Healthcare AI adoption mirrors the electronic medical record (EMR) revolution of the 2000s, but with potentially far greater impact on clinical practice.

• The Veterans Administration, which pioneered EMR development across 170 hospitals, now leads AI integration efforts by developing clinical predictive models and incorporating results into routine patient care.
• Future AI processing of patient records may resemble “a Google search rather than a traditional medical chart—pulling together all of the relevant information with a single click.”

Critical limitations: AI systems frequently generate false information through “hallucinations,” creating serious risks for patient care.

• These fabricated “factual” statements, including invented references and fake data, occur 40-75% of the time and appear to be increasing with each new AI version.
• AI models perform poorly at predicting low-frequency medical events like suicide, generating disruptive false alarms that vastly outnumber true positives.
• Studies suggest AI-based healthcare searches may produce massive amounts of disinformation, including convincing but fabricated medical images.

How it’s being implemented: Healthcare systems are developing safeguards and best practices for clinical AI deployment.

• AI predictive models require annual recalibration as they “drift” over time, losing accuracy without regular updates.
• “Ambient listening” technology uses AI to record, transcribe, and analyze patient-clinician conversations, automating documentation to free clinicians from keyboard work.
• AI systems increasingly search patient EMRs for relevant symptoms and link them to appropriate scientific literature.

Why this matters: The integration represents a pivotal moment where AI could either fulfill EMRs’ unrealized promise or amplify existing healthcare technology problems.

• Current EMR systems often fail to transfer records between institutions and perpetuate errors through copy-paste documentation practices.
• Clinical leaders worry about uncritical overreliance on AI and misrepresentation of risks and benefits to patients.
• Success depends on learning from EMR implementation challenges while addressing AI’s unique risks of generating false information.