NVIDIA Avatar Cloud Engine

NVIDIA Avatar Cloud Engine (ACE) is a suite of technologies that enables the creation of lifelike digital humans, AI non-player characters (NPCs), and interactive avatars using generative AI. Launched on June 2, 2024, ACE combines NVIDIA's advanced AI models and microservices to deliver high-quality, real-time performance across various applications, including gaming, customer service, and video conferencing.

Key features:
- State-of-the-art models and microservices: NVIDIA Riva for speech recognition, text-to-speech, and translation; NVIDIA NeMo for customizable generative AI models; NVIDIA Audio2Face for lifelike facial animation; and NVIDIA RTX for advanced rendering.
- Flexible deployment options: ACE models can run in the cloud or locally on NVIDIA RTX AI PCs. NVIDIA Inference Microservices (NIM) provide optimized containers for easy deployment.
- Safe and consistent results: AI models are trained on responsibly licensed data, with fine-tuning and guardrails to maintain relevance and appropriateness.

How it works:
ACE enables developers to create NPCs with natural language interactions for gaming, as demonstrated in the Covert Protocol sample application using Unreal Engine 5. NVIDIA Tokkio, built with ACE, provides AI-powered customer service capabilities across various industries. NVIDIA Maxine integrates real-time translation, text-to-speech, live portrait animation, and eye contact features for enhanced video conferencing.

Integrations:
- Gaming: Integration with Unreal Engine 5 and partnerships with game developers like Studio Wildcard for Project G-Assist.
- Customer Service: Collaboration with companies like Hippocratic AI and UneeQ for healthcare and retail applications.
- Video Conferencing: Integration with NVIDIA Maxine for enhanced communication features.

Use of AI:
NVIDIA ACE leverages generative AI models trained on NVIDIA supercomputers, including NVIDIA Riva for speech and language processing, NVIDIA NeMo for language generation and customization, and NVIDIA Audio2Face for facial animation.

AI foundation model:
The specific foundation model used by ACE is not mentioned in the provided information.

How to access:
ACE is available as a web app, API, and SDK. Developers can apply for early access to new ACE features, including NeMoTron-3 4.5B SLM, Speech Live Portrait, and Voice Font.