False legal citations expose the risks of generative AI in law

A federal judge has uncovered a troubling example of AI misuse in the legal system, revealing how generative AI can introduce fictional cases and misrepresentations into court documents. The case involving MyPillow CEO Mike Lindell highlights growing concerns about AI hallucinations in high-stakes legal proceedings, underscoring the need for proper oversight and verification when attorneys employ these increasingly accessible tools.

The big picture: A lawyer representing MyPillow CEO Mike Lindell admitted to using artificial intelligence to draft a legal brief containing approximately 30 defective citations, including completely fictional cases.

Key details: US District Judge Nina Wang identified numerous serious defects in the opposition brief filed by attorneys Christopher Kachouroff and Jennifer DeMaster.

• The problems included misquotes of cited cases, misrepresentations of legal principles, false statements about binding authorities, and “most egregiously, citation of cases that do not exist.”
• The defective brief was submitted February 25 in a defamation case brought by former Dominion Voting Systems employee Eric Coomer against Lindell.

Attorney admissions: Lead counsel Christopher Kachouroff acknowledged using generative AI to create the brief during an April 21 hearing.

• Kachouroff claimed he had personally outlined and drafted a brief before utilizing AI, but admitted after questioning that he failed to verify the citations.
• Both Kachouroff and DeMaster signed the problematic brief.

What’s next: Judge Wang has ordered the attorneys to explain themselves by May 5 or face potential sanctions.

• The lawyers must address “the circumstances surrounding the preparation of the Opposition,” including whether their clients were informed about and approved the use of generative artificial intelligence.
• They must also show cause as to why they shouldn’t face sanctions or be referred to disciplinary proceedings for violations of professional conduct rules.