Tianshou
An elegant PyTorch-based deep reinforcement learning library with modular APIs for both research and application development.
What is Tianshou?
Tianshou is a deep reinforcement learning library built on PyTorch that provides a modular framework for developing, training, and evaluating RL agents. It solves the problem of complex, inflexible codebases in RL by offering both user-friendly high-level APIs for applications and hackable low-level interfaces for algorithm research, supporting a wide range of algorithms from online DQN to offline CQL.
Reinforcement learning researchers seeking a flexible, type-safe library for algorithm development, and practitioners needing a high-performance, easy-to-use toolkit for applying RL to custom environments.
Developers choose Tianshou for its clean separation between algorithms and policies, extensive algorithm coverage, and dual API design that balances ease of use with research flexibility, all while maintaining high software engineering standards with thorough testing and documentation.
Overview
An elegant PyTorch deep reinforcement learning library.
Use Cases
Best For
- Implementing and experimenting with novel reinforcement learning algorithms
- Training RL agents on custom environments with vectorized parallelization
- Developing applications using offline reinforcement learning methods
- Multi-agent reinforcement learning research and prototyping
- Reproducible RL experiments with rigorous testing and benchmarking
- Educational purposes for learning deep RL concepts with clean PyTorch code
Not Ideal For
- Projects relying on non-Gymnasium environment standards or legacy reinforcement learning frameworks
- Teams needing a production-stable API without breaking changes, especially after major version updates
- Beginners or educators seeking plug-and-play RL solutions with minimal configuration and drag-and-drop interfaces
- Applications requiring extensive pre-trained models or industry-specific adaptations out-of-the-box
Pros & Cons
Pros
Offers both a high-level API for easy application development and a procedural API for hackable algorithm research, as demonstrated in the quick start examples with CartPole.
Implements over 30 state-of-the-art RL algorithms including DQN variants, PPO, SAC, and offline methods like CQL, covering online, offline, and multi-agent scenarios.
Supports vectorized environments and EnvPool integration for accelerated training, with numba-optimized operations for experience replay and GAE, ensuring high-speed execution.
Maintains rigorous testing with full agent training procedures, type hints, and comprehensive documentation, as highlighted in the comparison table with other RL platforms.
Cons
Version 2 is a complete overhaul that is not backwards compatible, requiring migration efforts for existing projects, as explicitly noted in the README's change log warning.
Requires poetry for full feature installation and managing extras for different environment types like mujoco or atari, which can be cumbersome compared to simpler pip-based setups.
The procedural API demands deeper understanding of RL concepts and PyTorch, making it less accessible for newcomers despite the high-level API's ease of use.
Multi-agent RL features are labeled as experimental in the documentation, meaning they may be less stable or well-documented compared to core algorithms.
Frequently Asked Questions
Related Projects
Stable Baselines3PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
GymnasiumAn API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
DopamineDopamine is a research framework for fast prototyping of reinforcement learning algorithms.
cleanrlHigh-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.