BRAX

Brax simulates millions of physics steps per second on TPUs and scales across multiple accelerators, enabling rapid experimentation at scale without complex infrastructure.

The engine supports gradient-based optimization, allowing learning algorithms like analytic policy gradients to leverage simulator gradients for efficient training and optimization.

Offers four interchangeable pipelines (MJX, Generalized, Positional, Spring) with a unified API, facilitating transfer learning and diverse simulation needs in a single framework.

Includes baseline algorithms such as PPO, SAC, and ARS, with training that can complete in seconds to minutes, streamlining reinforcement learning workflows directly within the engine.