The State of Simulation for Physical AI: An Overview

The primary challenge in physical AI is data availability, as robots cannot leverage internet-scale datasets like LLMs. Simulation bridges this gap by generating large amounts of photorealistic, physically grounded data at low cost. Modern simulators are now integrated into the model development loop for tasks such as perception dataset generation, RL training, policy evaluation, and sim-to-real transfer. Key engines include MuJoCo, which focuses on precise dynamics and contact-rich motion; MuJoCo Warp, a GPU-accelerated variant for batched simulation; NVIDIA Isaac Sim, an Omniverse-based framework for high-fidelity physics and rendering; and Isaac Lab, a lightweight multi-backend robot learning framework. The Newton physics engine, developed by NVIDIA, Google DeepMind, and Disney Research, is open-source and GPU-accelerated, providing multiple solvers for different physical systems. Newton is integrated into Isaac Lab and supports MuJoCo Warp as a backend. The ecosystem is fragmenting into a layered stack with shared infrastructure, much of it open-source.