AI’s rapid rise in healthcare sparks urgent calls for oversight

Health care leaders gathered at a recent Newsweek virtual event to address the critical challenge of implementing AI in medicine while maintaining patient safety. As artificial intelligence rapidly advances in health care settings, industry experts emphasized the need for robust governance frameworks, transparent oversight, and proactive quality assurance measures to prevent bias and errors in AI systems that could affect patient outcomes. This evolving landscape requires health organizations to balance innovation with careful risk management as they navigate the complexities of AI deployment in clinical environments.

The big picture: Health care organizations are simultaneously adopting AI tools while creating governance frameworks to ensure these technologies remain safe and effective over time.

• Dr. Brian Anderson, CEO of the Coalition for Health AI, highlighted that unlike traditional medical tools, AI systems change over time, potentially degrading in performance or experiencing drift.
• “We’re building this plane as we’re flying, so there is a real urgency to make sure these models and these tools are safe and that we’re managing them robustly and appropriately,” Anderson explained.

Governance approaches: Health care experts advocate for centralized oversight combined with implementation-specific monitoring that leverages local expertise.

• Dr. Michael Pencina of Duke Health emphasized the importance of an “umbrella oversight that sets the standards” alongside implementation-specific monitoring for individual algorithms.
• Dr. Andreea Bodnari, founder of Alignmt.AI, noted that the healthcare industry has an opportunity to introduce “proactive quality assurance for care delivery” through AI governance.

Transparency initiatives: The Coalition for Healthy AI promotes “model cards” that function like nutrition labels for AI tools, detailing their composition and performance characteristics.

• These transparency measures aim to build trust with physicians and patients by clearly documenting how AI systems operate and what their limitations might be.
• The approach parallels food product labeling, providing users with standardized information about what’s “inside” the AI tools they’re using.

Legal landscape: The regulatory and liability framework for healthcare AI remains in development, creating uncertainty for organizations implementing these technologies.

• Dr. Danny Tobey, chair of DLA Piper’s AI & Data Analytics Practice, noted that legal answers will emerge through “litigation and regulation and legislation,” creating a period of uncertainty.
• Organizations must navigate this evolving landscape while still moving forward with beneficial AI implementations.

Practical recommendations: Experts offered straightforward advice for health systems beginning their AI governance journey.

• Organizations should start by taking inventory of existing AI solutions already in use across their systems.
• Governance approaches should be appropriately scaled to available resources and risk levels, with more scrutiny allocated to high-risk applications.
• Health systems should avoid overcomplicated governance structures that might impede beneficial innovation.