AI “Coach” Detects Hallucinations, Boosts Accuracy and Safety

A new AI model called Lynx, developed by Patronus AI, aims to detect and explain hallucinations produced by large language models (LLMs), offering a faster, cheaper, and more reliable way to catch AI mistakes without human intervention.

Addressing the challenge of AI hallucinations: Patronus AI’s founders, ex-Meta AI researchers Anand Kannappan and Rebecca Qian, recognized the need for a solution to the problem of AI models confidently making factual errors:

• Kannappan and Qian spoke with numerous company executives who expressed concerns about launching AI products that could make headlines for the wrong reasons due to AI hallucinations.
• Lynx is designed to act as a “coach” for other AI models, guiding them to be more accurate and helping companies uncover hallucinations during the development phase rather than after product launch.

Lynx’s approach to detecting hallucinations: Patronus AI fine-tuned Meta’s advanced large language model, Llama 3, by training it on 2,400 examples of hallucinations and their corresponding correct responses:

• This approach provides Lynx with additional context to reason why an answer is wrong, making it more effective at catching similar mistakes compared to general-purpose models like GPT-4.
• Lynx outperforms most other AI models in detecting hallucinations, particularly in legal, financial, and medical domains, according to the company’s HaluBench benchmark.

Patronus AI’s other AI evaluation tools: The company has developed several tools to assess the performance and safety of AI models:

• Copyright Catcher detects when popular AI models produce copyrighted content, such as paragraphs from published books.
• FinanceBench evaluates how well LLMs answer financial queries, while Enterprise PII helps companies detect if AI models are exposing sensitive and confidential information.
• Simple Safety assesses LLMs for safety risks related to producing harmful responses on topics like suicide, child abuse, and fraud.

Broader implications for AI oversight: Patronus AI’s mission is to ensure that LLMs do not produce inaccurate or harmful results that people may rely on, leading to the spread of misinformation:

• The development of Lynx highlights the growing need for scalable oversight of AI systems that can outperform humans, which requires powerful AI models to evaluate other AI models.
• As more companies deploy AI applications, tools like Lynx could become increasingly important in maintaining the accuracy and safety of these systems, helping to prevent potential reputational damage and the spread of misinformation.