×
Cohere launches Aya Vision: Multilingual AI models now support 23 languages
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

Cohere For AI’s new Aya Vision models represent a significant breakthrough in multilingual capabilities for vision-language AI systems. The 8B and 32B parameter models support 23 languages, addressing a critical gap in the AI landscape where most vision models have focused primarily on English content. This advancement could democratize access to multimodal AI technologies across diverse linguistic communities and applications, potentially transforming how visual content is processed across global markets.

The big picture: Cohere For AI has released Aya Vision, a new family of open-weight vision-language models designed to bring robust multilingual capabilities to multimodal AI systems.

  • The models build upon the foundation established by the company’s Aya Expanse multilingual language models while extending capabilities to visual content understanding across 23 languages.
  • Available in both 8B and 32B parameter versions, Aya Vision represents a targeted effort to address language diversity in the rapidly evolving multimodal AI space.

Key technical details: Aya Vision employs SigLIP2-patch14-384 as its vision encoder and incorporates advanced processing techniques for efficient image handling.

  • The architecture uses dynamic image resizing and Pixel Shuffle for optimization, striking a balance between performance and computational efficiency.
  • Training followed a two-stage approach: vision-language alignment followed by supervised fine-tuning, with special attention to comprehensive language coverage.

Innovative approaches: The development team implemented several novel techniques to enhance multilingual performance in visual contexts.

  • Synthetic data annotations and translation processes were employed to overcome limitations in available training data for non-English languages.
  • The team leveraged multimodal model merging methods to efficiently scale capabilities from the 8B to 32B parameter versions.

Performance highlights: The models demonstrate strong capabilities across various vision-language tasks while maintaining multilingual support.

  • Aya Vision excels in image captioning, visual question answering, and text generation tasks that involve visual reasoning.
  • The models outperform several larger competitors specifically in multilingual evaluations, suggesting efficient architecture design.

Practical applications: Users can access and implement Aya Vision through several channels for immediate experimentation.

  • The models and associated resources, including the new AyaVisionBench evaluation dataset, are available through Hugging Face repositories.
  • Practical implementations include a Hugging Face Space demo, WhatsApp integration, and a Colab example to help users get started quickly.

Why this matters: Bringing robust multilingual capabilities to multimodal AI addresses a significant equity gap in current vision-language technologies.

  • Most existing vision-language models have focused primarily on English, limiting access and utility for billions of non-English speakers worldwide.
  • Democratizing these capabilities could accelerate AI adoption and application development across diverse global communities and markets.
A Deepdive into Aya Vision: Advancing the Frontier of Multilingual Multimodality

Recent News

Unpublished AI system allegedly stolen by synthetic researcher on GitHub

The repository allegedly contains an unpublished recursive AI system architecture with suspicious backdated commits and connection to a potentially synthetic researcher identity with falsified credentials.

The need for personal AI defenders in a world of manipulative AI

Advanced AI systems that protect users from digital manipulation are emerging as essential counterparts to the business-deployed agents that increasingly influence consumer decisions and behavior.

AI excels at identifying geographical locations but struggles with objects in retro games

Modern AI systems show paradoxical visual skills, excelling at complex geographic identification while struggling with simple pixel-based game objects.