NVIDIA releases four open Cosmos 3 models to simulate driving and robots

NVIDIA has released four open-weight checkpoints in its Cosmos 3 family, built for interactive world simulation in autonomous driving and robotics, in a post published on Hugging Face on 5 November 2026. All four are post-trained versions of Cosmos 3 Nano with 16B trainable parameters, and they ship under the OpenMDW1.1 licence, which permits commercial and non-commercial use under its terms.

A world model, in this sense, is a generative model that predicts what a scene will look like next, given what has happened and what an agent does. The new checkpoints turn driving controls, physics-simulator outputs or robot actions into generated camera video and, for driving, LiDAR (laser-based 3D sensing). NVIDIA’s documentation lists the release as version 3.2, “Interactive World Simulation with Cosmos for Physical AI”, including example model checkpoints and training recipes.

What NVIDIA released on 5 November

The four new checkpoints each target one simulation job:

• Cosmos3-Nano-Transfer-Auto: generates multi-view driving data with varied weather, lighting and time of day.
• Cosmos3-Nano-Sim-Auto: streams synchronized camera and LiDAR observations for driving simulation.
• Cosmos3-Nano-Sim-Transfer: turns simulator controls and a text prompt into photorealistic video.
• Cosmos3-Nano-Sim-Bimanual: predicts video from robot or camera actions.

NVIDIA frames the release as a shared starting point: open weights and post-training recipes for adapting models, comparing methods and contributing improvements. The post lists Ming-Yu Liu among its authors; he is vice president of Cosmos Lab at NVIDIA.

Checkpoint	Input	Output	Mode
Transfer-Auto	Per-camera World Scenario Map controls, text, optional RGB images, HD-map range controls for LiDAR	Video across up to 11 camera views, joint RGB–LiDAR	Offline data generation
Sim-Auto	Per-camera captions, synchronized scenario controls, optional RGB images	Successive chunks of camera video and LiDAR	Streaming, few denoising steps
Sim-Transfer	Depth, segmentation, edge or blur controls plus a text prompt	Color (RGB) video	Rendering layer over a physics simulator
Sim-Bimanual	An image plus robot or camera actions	Predicted video	Interactive two-arm simulation

Table compiled from NVIDIA’s announcement.

How the Cosmos 3 driving models work

Transfer-Auto is a bidirectional model for offline synthetic driving data generation. It produces synchronized video across up to 11 camera views, including fisheye lenses, and supports joint RGB–LiDAR generation. Per-camera World Scenario Map (WSM) controls set the scene structure and traffic motion, while text prompts guide appearance; optional initial RGB images anchor how the scene looks. LiDAR generation additionally uses aligned HD-map range-map controls.

NVIDIA positions Transfer-Auto as the base for adapting Cosmos to a team’s own sensor rig and driving domain: post-train it on your own data to match a camera layout, LiDAR configuration and target environment. The adapted model can then be used directly for offline data generation, or go through causal training and few-step distillation for fast streaming simulation.

That distillation path is how NVIDIA built Sim-Auto, the causally distilled streaming variant of Transfer-Auto. Distillation here means training a model to reach similar output in far fewer generation steps. Sim-Auto generates successive small chunks of synchronized camera video and LiDAR in a few denoising steps, using its own generated history plus updated driving controls to support long, interactive rollouts. NVIDIA notes that closed-loop interaction, where a driving policy’s choices feed back into the scene, requires a simulation runtime and a WSM renderer to update those controls from driving inputs. The model alone is not a complete driving simulator.

NVIDIA’s stated reasoning is that in autonomous driving, appearance and geometry must agree, and generating camera and LiDAR together connects visual observations with 3D sensing. Scene and traffic-signal controls let researchers vary road layouts and signal states, then study how a driving policy responds.

What the robotics checkpoints add

The two robotics-oriented models take different routes to the same goal: a simulated environment that reacts to what a robot does.

Sim-Transfer: a renderer on top of a physics simulator

Sim-Transfer leaves physics to an existing simulator, which computes motion, contacts and physical state. The model takes the simulator’s spatial controls (depth, meaning relative distance from the camera; segmentation, meaning regions and object boundaries; edges, meaning scene contours; and blur, giving coarse appearance and layout) and a text prompt, then renders color observations that a policy can consume. NVIDIA describes the loop as action, simulator update, spatial controls, generated RGB, next policy action, with physics and ground-truth state staying in the simulator.

Sim-Bimanual: predicting video from robot actions

Sim-Bimanual simulates how an environment changes in response to a two-armed robot’s actions. It supports interactive two-arm simulation for YAM and AgiBot embodiments, controlled either by a policy or by human-operated joysticks. A user starts with an image, moves the robot or the camera, and the model generates the resulting video.

Where this fits in the Cosmos 3 family

The new checkpoints build on a base model NVIDIA introduced earlier in 2026. According to aivancity’s coverage, Cosmos 3 was unveiled at GTC Taipei 2026 alongside the Isaac GROOT humanoid robot, and NVIDIA presents it as the first fully open “omnimodel” for physical AI, meaning AI that perceives and acts in the physical world. NVIDIA’s release notes describe version 3.0 as merging separate models into a single suite of omnimodal world models built on a unified Mixture-of-Transformers architecture. Earlier Cosmos generations were split across separate repositories, Predict1, Transfer1 and Reason1.

The same coverage describes two initial versions: Super, for applications needing high physical accuracy such as industrial robotics and autonomous driving, and Nano, for faster response times and lower computing costs, with an Edge version announced for local devices. NVIDIA’s release notes now list Cosmos3-Edge as released in version 3.1, as a 4B tier model for on-device, real-time deployment. All four new checkpoints derive from the Nano tier.

On training data, aivancity reports NVIDIA’s figure of nearly 20,000 billion tokens, including nearly one billion images, about 400 million real and synthetic videos, ambient audio, text and traces of actions by humans and robots. NVIDIA also claims some training phases that once took months can now be completed in a few days, according to the same report. Neither figure is independently verified in the sources.

Access, licence and tooling

The checkpoints are announced on Hugging Face, with links to download them, a technical report and demos. The licence is OpenMDW1.1, and NVIDIA says it permits commercial and non-commercial use under its terms. Teams planning commercial products should read those terms rather than rely on the summary.

For each model, NVIDIA provides an “open skill” that a user can point a coding agent to in order to run inference, as well as a notebook for those who prefer one. The documentation release notes say the 3.2 release includes training recipes alongside the example checkpoints. The sources do not describe a hosted API for these checkpoints or hardware requirements for running them.

What it means for builders, and the limits

For teams working on driving or robot-learning software, the practical value is a set of open starting points that can be adapted rather than trained from scratch. NVIDIA’s recommended path is explicit: adapt Transfer-Auto to your rig first, then distill it into a Sim-Auto-style streaming model. Sim-Transfer offers a way to add photorealistic visuals to an existing physics simulator without replacing it. NVIDIA also argues that streaming generation links video prediction to ongoing interaction, since each observation can shape the next action.

Use cases highlighted in coverage include generating rare or dangerous scenarios, such as collisions, industrial accidents and mechanical failures, that are hard or risky to reproduce for real-world training.

The limits are stated by NVIDIA itself:

These problems remain hard: the work ahead is to measure where the models faithfully support policy learning and evaluation, understand their limits, and improve where they fall short.

Several open questions follow from the sources:

• No benchmarks in the announcement. The post does not report evaluation numbers or comparisons with other world models, and there is no third-party evaluation in the sources.
• Simulation fidelity. Whether a policy trained or tested in these generated worlds transfers to real vehicles and robots is exactly what NVIDIA says still needs measuring.
• Extra infrastructure. Closed-loop driving simulation needs a separate simulation runtime and WSM renderer, and Sim-Transfer depends on an external physics simulator.
• Narrow robot support. Sim-Bimanual names only YAM and AgiBot embodiments; other robots would presumably need post-training.

What to watch next

The key signal will be evidence that these models support policy learning and evaluation, which NVIDIA names as its own next task. Independent tests of how well Sim-Auto holds up over long rollouts, and of whether teams succeed in adapting Transfer-Auto to their own sensor rigs, will show how useful the open recipes are in practice. Also watch whether NVIDIA publishes post-trained checkpoints from the Super tier or for the 4B Edge model, and whether more robot embodiments are added to the bimanual simulator.