AI search tools provide wrong answers up to 60% of the time despite growing adoption

AI-powered search tools are rapidly replacing traditional search engines for many users, with nearly one-third of US respondents now using AI instead of Google according to research from Future. However, recent testing reveals significant accuracy problems across major AI search platforms, raising serious questions about their reliability for information retrieval. This shift in search behavior is occurring despite concerning evidence that even the best AI search tools frequently provide incorrect information, fail to properly cite sources, and repackage content in potentially misleading ways.

The big picture: Independent testing shows AI search tools are far from ready to replace traditional search engines, with accuracy rates that should concern regular users.

• The Tow Center for Digital Journalism found major AI models gave incorrect answers to more than 60% of queries across their testing of eight platforms including ChatGPT, Perplexity, Copilot, Grok, and Gemini.
• Even Perplexity, marketed specifically as a research tool and the best performer in testing, still provided wrong answers 37% of the time.
• Grok performed particularly poorly, with an alarming 94% failure rate in the researchers’ evaluation.

Behind the numbers: The accuracy problems stem from several fundamental issues in how AI search tools process and present information.

• AI models frequently hallucinate information, making up facts that don’t appear in their source material.
• Even when not entirely wrong, these tools often repackage content in ways that distort or oversimplify the original information.
• The conversational interface of AI chatbots masks serious underlying problems with information quality, creating false confidence in their outputs.

Why this matters: AI search tools fundamentally alter the relationship between users, information providers, and content creators in problematic ways.

• Traditional search engines function as intermediaries that direct users to original sources, while AI tools parse and repackage information themselves, cutting off traffic to the original creators.
• Poor citation practices make verification difficult, with ChatGPT often linking to incorrect articles, general homepages, or skipping citations entirely.
• This approach both undermines publishers’ business models and makes fact-checking more burdensome for users.

The bottom line: Despite growing adoption of AI search tools and their integration into traditional search results, current AI models require significant human oversight and verification to be trustworthy information sources.

• Using AI for search may actually create more work for users who need to verify results through traditional search engines anyway.
• While tools like Perplexity perform better than general-purpose AI chatbots, even the best current options fail frequently enough to warrant caution.
• The convenience of AI-generated summaries comes with significant trade-offs in accuracy and transparency that users should carefully consider.