When no does in fact mean yes: AI models fail to understand Persian ritual politeness

New research reveals that mainstream AI language models from OpenAI, Anthropic, and Meta fail to understand taarof—a Persian cultural practice of ritual politeness where “no” often means “yes”—correctly navigating these social situations only 34-42% of the time compared to 82% for native Persian speakers. This cultural blindness in AI systems could lead to significant misunderstandings in global business, diplomatic, and social contexts as these models increasingly facilitate cross-cultural communication.

What you should know: The study, conducted by Nikta Gohari Sadr of Brock University along with researchers from Emory University, tested major AI models including GPT-4o, Claude 3.5 Haiku, Llama 3, DeepSeek V3, and Dorna across taarof scenarios.

• Taarof governs countless daily interactions in Persian culture through ritualized exchanges of offers, refusals, and counter-refusals—like when an Iranian taxi driver says “Be my guest this time” but expects you to insist on paying three times before accepting payment.
• The research introduces “TAAROFBENCH,” the first benchmark specifically designed to measure AI systems’ ability to reproduce this intricate cultural practice.

The big picture: AI models default to Western-style directness, completely missing cultural cues that govern everyday interactions for millions of Persian speakers worldwide.

• A response can register as 84.5% “polite” by Western standards while only meeting Persian cultural expectations 41.7% of the time—a 42.8 percentage point gap.
• Common AI failures include accepting offers without initial refusal, responding directly to compliments rather than deflecting them, and making direct requests without hesitation.

How language affects performance: When researchers switched from English to Persian prompts, AI accuracy improved dramatically across most models.

• DeepSeek V3’s accuracy jumped from 36.6% to 68.6% when prompted in Persian rather than English.
• GPT-4o showed similar gains, improving by 33.1 percentage points when using Persian prompts.
• The language switch apparently activated different Persian-language training data patterns that better matched these cultural encoding schemes.

Human benchmark comparison: The study tested 33 participants across different cultural backgrounds to establish performance baselines.

• Native Persian speakers achieved 81.8% accuracy, setting the performance ceiling.
• Heritage speakers (Persian descent but English-educated) reached 60% accuracy.
• Non-Iranians scored 42.3%, nearly matching AI model performance and showing similar cultural blind spots.

Gender bias patterns: All tested AI models showed concerning gender-specific response patterns when navigating taarof scenarios.

• Models received higher cultural accuracy scores when responding to women than men, with GPT-4o showing 43.6% accuracy for female users versus 30.9% for male users.
• Models frequently supported responses using gender stereotypes, stating “men should pay” or “women shouldn’t be left alone” even when taarof norms apply equally regardless of gender.
• “Despite the model’s role never being assigned a gender in our prompts, models frequently assume a male identity and adopt stereotypically masculine behaviors in their responses,” the researchers noted.

Training solutions show promise: Researchers tested whether AI models could learn taarof through targeted training approaches.

• Direct Preference Optimization (a training technique where AI models learn to prefer certain responses over others through example pairs) doubled Llama 3’s performance from 37.2% to 79.5% accuracy on taarof scenarios.
• Supervised fine-tuning produced a 20% improvement in cultural understanding.
• Simple in-context learning with just 12 examples improved performance by 20 percentage points.

Why this matters: Cultural missteps in high-consequence settings can derail negotiations, damage relationships, and reinforce stereotypes as AI systems become more prevalent in global contexts.

• The findings suggest AI models possess many untested cultural blind spots that could have significant impacts when facilitating cross-cultural communication and translation.
• The methodology offers a template for evaluating cultural understanding in other traditions not well-represented in Western-dominated AI training datasets.