NVIDIA Unveils Cosmos 3 Omnimodel to Accelerate Physical AI Training

NVIDIA unveiled Cosmos 3 at GTC Taipei as an open omnimodel designed to accelerate physical AI by bundling vision reasoning, world generation and action prediction into a single model family. The company says Cosmos 3 can reduce physical AI training and evaluation cycles from months to days by providing pretrained multimodal capabilities, a change that would shorten development loops for teams building robots, autonomous vehicles and other systems that must perceive, reason and act in the real world.

Cosmos 3 is built on a mixture — of-transformers architecture that pairs a reasoning transformer with an expert generation transformer. That design lets the model first infer object interactions, motion and spatiotemporal relationships and then generate video and action trajectories. The system natively supports multiple modalities — text, images, video, ambient sound — and produces action outputs, enabling unified reasoning and generation across sensing and acting modalities.

NVIDIA says Cosmos 3 was trained on one of the largest multimodal physical AI datasets, comprising billions of samples spanning text, images, video, sound and action trajectories. The company positions the model as a vision — language reasoning system, a video/world foundation model for environment simulation, and a backbone for training world action policies used in robotics and autonomous vehicles. Cosmos 3 Super for highest physics accuracy and generation quality, Cosmos 3 Nano for low-latency video and action reasoning, and an Edge edition coming soon for real-time inference at the edge.

Alongside the model, NVIDIA launched the Cosmos Coalition, a consortium of world — model builders and robotics labs that includes Agile Robots, Black Forest Labs, Generalist, LTX, Runway and Skild AI. Coalition members will contribute models, research and evaluation techniques while adopting Cosmos 3 technologies and using NVIDIA DGX Cloud infrastructure for large — scale training, according to the company.

On open-model leaderboards, NVIDIA reports Cosmos 3 leading across multiple physical AI benchmarks, citing top ranks on Artificial Analysis, Physics‑IQ, PAI‑Bench and R‑Bench for world generation; RoboLab and RoboArena for action policy; and VANTAGE‑Bench and TAR for vision understanding. Jensen Huang said, “The big bang of physical AI is just around the corner…The Cosmos 3 family of open, frontier omnimodels gives developers a generational leap in ability to build robots, autonomous vehicles and vision AI that perceive, reason, plan and act in the physical world.