AI chatbots fail to deliver reliable financial guidance

Leading AI chatbots are failing dramatically at financial advice, demonstrating how conversational AI systems can present dangerously incorrect information with an authoritative tone. A new study from the Walter Bradley Center for Natural and Artificial Intelligence tested four top large language models with basic financial questions, revealing significant deficiencies in mathematical accuracy and financial reasoning that could mislead users who trust these systems for important financial decisions.

The big picture: AI researchers tested ChatGPT-4o, DeepSeek-V2, Grok 3 Beta, and Gemini 2 with 12 finance questions, finding all models performed poorly despite their confident conversational style.

• None of the chatbots scored higher than 5 out of a possible 12 points, with GPT-4o leading at 5.0, followed by DeepSeek (4.0), Grok (3.0), and Gemini trailing significantly at 1.5.
• The researchers used a straightforward scoring system: 0 for completely incorrect analysis, 0.5 for correct financial reasoning with math errors, and 1 for fully correct responses.

Key details: The researchers found these AI systems offer a “reassuring illusion of human-like intelligence” reinforced by a conversational style that can mask fundamental errors.

• In one striking example, Elon Musk’s Grok calculated that $3,700 rent plus $200 in utilities equals $4,900 per month for a Caribbean rental property.
• This study mirrors findings from researcher Gary Smith’s 2023 assessment for the Journal of Financial Planning, which found earlier AI models similarly produced “seemingly authoritative but riddled with arithmetic and critical-thinking mistakes” responses.

Why this matters: The friendly, confident tone of AI chatbots creates a particularly dangerous dynamic where users might trust these systems with important financial decisions.

• The researchers concluded that “the real danger is not that computers are smarter than us, but that we think computers are smarter than us and consequently trust them to make decisions they should not be trusted to make.”