Anti-deepfake law may have been written by AI

Minnesota’s anti-deepfake election law faces scrutiny as a federal lawsuit uncovers potential AI-generated content in a legal affidavit meant to defend the legislation, raising questions about the reliability of expert testimony and the ironies of AI involvement in anti-AI legislation.

Key developments: The controversy centers on an affidavit submitted by Stanford Social Media Lab founding director Jeff Hancock in support of Minnesota’s law regulating deepfake technology in elections.

• The affidavit cites multiple academic studies that appear to be non-existent, including a purported 2023 study in the Journal of Information Technology & Politics
• These phantom citations show characteristics of AI hallucinations, where large language models generate false but convincing-sounding references
• Hancock has not responded to requests for comment about the authenticity of his submission

Legal implications: The discovery of potentially AI-generated content has prompted a strong response from attorneys challenging the state’s deepfake law.

• Lawyers representing state Rep. Mary Franson and conservative YouTuber Christopher Khols have filed a motion highlighting these discrepancies
• The legal team argues that the presence of AI hallucinations “calls the entire document into question”
• The filing specifically criticizes the affidavit’s lack of methodology and analytical logic

Technical context: AI hallucinations, a term describing when language models generate false but plausible-sounding information, have become a growing concern in professional and academic contexts.

• Large language models like ChatGPT can produce convincing but entirely fabricated academic citations
• These AI systems often create detailed but false references by combining elements of real academic writing
• The incident highlights the challenges of detecting AI-generated content in legal and professional documents

Unintended consequences: The use of potentially AI-generated content to defend an anti-deepfake law exposes the complex challenges of regulating artificial intelligence technologies.

• The situation demonstrates how AI can inadvertently infiltrate even efforts aimed at controlling its influence
• The controversy raises questions about the verification processes for expert testimony and legal declarations
• Minnesota Attorney General Keith Ellison’s office faces scrutiny over its vetting of expert submissions

Critical Analysis: The irony of potentially using AI-generated content to support anti-AI legislation underscores the growing difficulty of maintaining authenticity in legal and academic discourse as AI tools become more sophisticated and widely available. This incident may lead to increased skepticism of expert testimony and stricter verification requirements for legal submissions.