AI regulation in healthcare must also include algorithm oversight, researchers say

The increasing integration of AI and algorithmic tools in healthcare has prompted calls for comprehensive regulatory oversight to ensure patient safety and prevent discrimination.

Current regulatory landscape: The U.S. Office for Civil Rights has implemented a new Affordable Care Act rule that prohibits discrimination in patient care decision support tools, encompassing both AI and traditional algorithms.

• FDA oversight of AI-enabled medical devices has expanded significantly in recent years, reflecting the growing presence of artificial intelligence in healthcare
• Despite widespread use, clinical risk scores and decision support tools currently operate without dedicated regulatory supervision
• The new rule marks a significant step toward addressing potential biases in healthcare technology

Expert perspectives: Leading researchers from prestigious institutions including MIT, Boston University, and Equality AI are advocating for expanded regulatory frameworks in healthcare technology.

• MIT associate professor Marzyeh Ghassemi emphasizes the importance of the new rule as a crucial development in healthcare oversight
• Harvard Medical School’s Isaac Kohane points out that traditional clinical risk scores, while more transparent than AI algorithms, are still limited by the quality of their training data
• Equality AI CEO Maia Hightower acknowledges the complexities of regulating clinical risk scores while maintaining their necessity for ensuring fairness

Upcoming developments: The healthcare technology community is mobilizing to address regulatory challenges through collaborative efforts and dialogue.

• MIT’s Jameel Clinic will convene a regulatory conference in March 2025 to explore solutions and frameworks
• Stakeholders are working to balance innovation with patient protection in both AI and traditional algorithmic tools
• The incoming administration’s deregulatory stance may present additional challenges to implementing new oversight measures

Implementation challenges: The widespread adoption of clinical decision support tools across healthcare systems creates complex regulatory considerations.

• The sheer volume of existing clinical risk scores makes comprehensive oversight challenging
• Healthcare providers rely heavily on these tools for daily decision-making, necessitating careful regulatory implementation
• Ensuring transparency and non-discrimination must be balanced against maintaining operational efficiency

Future implications: The push for regulatory oversight of healthcare algorithms represents a critical turning point in medical technology governance, though success will depend on navigating political headwinds and practical implementation challenges while maintaining healthcare innovation and efficiency.