From language to cultural authenticity, AI adoption in the Global South faces challenges

Global South countries are actively reshaping the AI landscape by developing localized solutions that address their unique linguistic and cultural contexts, challenging the Western-centric approach of mainstream large language models. This movement represents a significant shift in AI development, with smaller, culturally-attuned models potentially offering more relevant solutions to regional challenges in healthcare, education, and environmental management than their larger, English-dominated counterparts.

The big picture: Despite the global race to develop increasingly powerful large language models (LLMs) like GPT-4 and Gemini, these systems perform poorly in non-Western languages and cultural contexts, limiting their utility for much of the world’s population.

• Current mainstream LLMs, trained predominantly on English and Western-centric data, struggle with grammar, vocabulary, and cultural nuances in languages such as Hindi, Arabic, and Xhosa.
• Testing by researchers has revealed major performance gaps in tasks like reasoning, summarization, and text classification for nearly all African languages evaluated through the AfroBench multi-task benchmark.

Why this matters: In a digital-first world dominated by English-language content and AI tools, there’s a real risk of cultural erasure, particularly affecting younger generations who may grow up disconnected from local history and cultural context.

• The development of localized AI systems represents an opportunity for communities in the Global South to preserve cultural heritage while addressing pressing societal challenges specific to their regions.

Key developments: Several grassroots initiatives are working to shift the AI development focus toward localized needs and capabilities, challenging the concentration of AI power within major technology companies.

• The Deep Learning Indaba, a non-profit founded in 2017, is building machine learning communities across Africa to ensure African researchers participate in global AI conversations.
• Masakhane, a grassroots community focused on natural language processing for African languages, drove the development of AfroBench, which evaluates LLM performance across 64 African languages.

Notable innovation: Researchers in the Global South are developing small language models (SLMs) with fewer than a billion parameters that are specifically optimized for local contexts and applications.

• These smaller models can outperform larger ones in specialized tasks like mathematical problem solving and code generation while requiring significantly fewer computational resources.
• The focus on SLMs promotes an entrepreneurial AI ecosystem centered on practical regional challenges in education, health, and environmental management.

Real-world applications: In India, researchers have identified potential for generative AI to support public mental health care, particularly in suicide prevention.

• LLM-based systems could help address challenges posed by India’s vast population, cultural diversity, shortage of trained professionals, and mental health stigma.
• However, current systems remain inadequate for such specialized, culturally sensitive applications, highlighting the need for locally-tailored solutions.

The path forward: Countries in the Global South have the opportunity to take charge of the AI agenda by leveraging their unique ideas, talent, and data to develop solutions that specifically address local contexts.

• Government support and appropriate ethical and governance frameworks are needed to ensure countries develop independent AI infrastructure.
• Rather than joining the global race for ever-larger models, researchers and developers in the Global South can focus on creating valuable curated datasets in native languages and determining AI research directions that will benefit their communities.