A C++-based high-performance parallel environment execution engine for vectorized reinforcement learning simulations.
EnvPool is a C++-based, high-performance parallel environment execution engine designed for reinforcement learning. It vectorizes environment simulations, allowing multiple environments to run concurrently in batched operations, drastically speeding up RL training loops. It solves the bottleneck of slow environment simulation in traditional Python-based RL setups.
Reinforcement learning researchers and engineers who need to train agents faster, particularly those working with large-scale experiments requiring high-throughput environment simulation. It's also valuable for developers integrating custom environments into accelerated RL pipelines.
Developers choose EnvPool for its exceptional performance gains (up to ~20x speedup), broad compatibility with major RL APIs and libraries, and its general-purpose design that works across various environment types unlike GPU-specific solutions.
C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Achieves up to 1 million Atari frames per second on 256 CPU cores, delivering ~20x speedup over Python subprocess-based vectorized environments as benchmarked.
Supports OpenAI Gym, DeepMind dm_env, and Gymnasium APIs, enabling seamless integration with existing RL codebases without modification.
Offers both synchronous and asynchronous execution, plus single-player and multi-player environment support, adaptable to diverse training pipelines.
Works with popular RL libraries like Stable-Baselines3 and Tianshou, with provided examples for quick adoption and benchmarking.
Installation requires handling quirks like Qt for Procgen on Linux and Mesa setup for MuJoCo rendering on Windows, adding complexity.
Integrating new environments necessitates C++ development via the provided API, which can be a hurdle for Python-centric teams.
Only supports pre-listed environment types; unsupported simulators require significant porting effort, limiting out-of-the-box usability.