NVIDIA Launches AI Microservices for Japan and Taiwan Markets

AI localization advances: NVIDIA has launched four new NIM microservices to accelerate the deployment of sovereign AI applications with enhanced cultural and language fluency in Japan and Taiwan.

• The new microservices support popular community models tailored to meet regional needs, improving user interactions through more accurate understanding and responses based on local languages and cultural heritage.
• This initiative aligns with the growing global trend of nations pursuing sovereign AI to ensure AI systems are in harmony with local values, laws, and interests.
• ABI Research projects that generative AI software revenue in the Asia-Pacific region alone is expected to surge from $5 billion in 2024 to $48 billion by 2030, highlighting the significant market potential.

Specialized language models: The new NIM microservices include region-specific models trained on local data to provide deeper understanding of cultural nuances and linguistic subtleties.

• Llama-3-Swallow-70B, trained on Japanese data, and Llama-3-Taiwan-70B, trained on Mandarin data, offer enhanced comprehension of local laws, regulations, and customs.
• The RakutenAI 7B family of models, built on Mistral-7B and trained on English and Japanese datasets, is available as two separate NIM microservices for Chat and Instruct functionalities.
• These models have demonstrated superior performance in regional language understanding, legal tasks, question-answering, and language translation compared to base LLMs like Llama 3.

Deployment and performance: The new NIM microservices, available with NVIDIA AI Enterprise, are designed for easy deployment and optimized performance.

• The microservices are optimized for inference using the NVIDIA TensorRT-LLM open-source library, providing up to 5x higher throughput for Llama 3 70B-based models.
• This optimization leads to lower operational costs and improved user experiences through decreased latency.
• The microservices are currently available as hosted application programming interfaces (APIs), allowing for straightforward integration into existing systems.

Real-world applications: Various organizations across industries are already leveraging these new NIM microservices to enhance their AI capabilities.

• The Tokyo Institute of Technology has fine-tuned Llama-3-Swallow 70B using Japanese-language data, emphasizing the importance of developing sovereign AI models that adhere to cultural norms.
• Preferred Networks, a Japanese AI company, is using the model to develop a healthcare-specific model trained on Japanese medical data, which has shown top performance on the Japan National Examination for Physicians.
• Chang Gung Memorial Hospital in Taiwan is building a custom AI Inference Service using Llama 3-Taiwan 70B to improve the efficiency of frontline medical staff and enhance patient care.
• Taiwanese companies like Pegatron, Chang Chun Group, and Unimicron are adopting Llama 3-Taiwan 70B for various applications in manufacturing, operations, and business processes.

Customization capabilities: NVIDIA AI Foundry provides a platform for enterprises to further customize these regional models for specific business needs.

• The AI Foundry platform includes popular foundation models, NVIDIA NeMo for fine-tuning, and dedicated capacity on NVIDIA DGX Cloud.
• This full-stack solution allows developers to create customized foundation models packaged as NIM microservices, ensuring culturally and linguistically appropriate results for their users.
• The platform also provides access to NVIDIA AI Enterprise software, offering security, stability, and support for production deployments.

Future implications: The launch of these localized AI microservices represents a significant step towards more culturally attuned and efficient AI systems.

• As more countries invest in sovereign AI infrastructure, these tools will likely play a crucial role in developing AI applications that respect local values and regulations.
• The ability to fine-tune models for specific industries and use cases could lead to more specialized and effective AI solutions across various sectors.
• This development may also encourage further innovation in language-specific AI models, potentially leading to more diverse and inclusive AI technologies globally.