NVIDIA’s Omniverse: How OpenUSD workflows advance physical AI for robotics and vehicles

Core innovation: NVIDIA recently unveiled Cosmos, a platform of generative world foundation models designed to accelerate the development of physical AI systems through advanced simulation and synthetic data generation.

• The platform includes state-of-the-art models, tokenizers, guardrails, and video processing capabilities specifically built for physical AI applications
• Cosmos enables the creation of detailed virtual environments that incorporate real-world physics, spatial relationships, and cause-and-effect principles
• The technology is particularly focused on applications in robotics, autonomous vehicles, and vision AI systems

Technical capabilities: When integrated with NVIDIA Omniverse and powered by OpenUSD, Cosmos creates a powerful synthetic data generation engine for AI development.

• Developers can create 3D scenarios in Omniverse and use Cosmos to generate controlled videos and variations
• The platform significantly reduces the time and resources typically required for physical AI development by generating large amounts of photoreal, physics-based synthetic data
• OpenUSD technology ensures seamless integration and consistent representation of data across different scenarios

Industry adoption: Major companies across robotics, automotive, and ridesharing sectors are implementing Cosmos for various applications.

• Leading robotics companies including 1X, Agile Robots, and Figure AI are among early adopters
• Uber is partnering with NVIDIA to accelerate autonomous mobility development
• KION Group is utilizing the technology for warehouse automation applications

Key applications: The platform supports three main use cases that demonstrate its practical implementation.

• Humanoid robotics: The NVIDIA Isaac GR00T Blueprint generates synthetic motion datasets for training humanoid robots
• Autonomous vehicles: Integration with Omniverse Sensor RTX APIs enables comprehensive testing and training scenarios
• Industrial automation: The Mega Blueprint facilitates development and testing of robot fleets in virtual environments before physical deployment

Looking ahead: As physical AI continues to evolve, platforms like Cosmos will be crucial in bridging the gap between virtual simulation and real-world implementation.

• The technology’s ability to generate massive amounts of synthetic data could significantly accelerate the development timeline for autonomous systems
• The open model license availability on Hugging Face and NVIDIA NGC catalog suggests a commitment to fostering broader innovation in the field
• The success of these implementations will likely influence future developments in physical AI training methodologies and simulation technologies