Imitation
Clean PyTorch implementations of imitation and reward learning algorithms for reinforcement learning.
What is Imitation?
Imitation is a Python library that provides clean, production-ready implementations of imitation learning and reward learning algorithms. It solves the problem of training reinforcement learning agents efficiently by leveraging expert demonstrations, human preferences, or inferred reward functions, rather than relying solely on trial-and-error. The library includes algorithms like Behavioral Cloning, GAIL, AIRL, and Inverse Reinforcement Learning, all built with PyTorch.
Reinforcement learning researchers and practitioners who need reliable baselines for imitation learning, as well as developers working on AI systems that learn from human demonstrations or preferences, particularly in robotics and game AI.
Developers choose Imitation for its high-quality, well-documented implementations that are rigorously tested and benchmarked, offering a standardized toolkit that reduces implementation errors and accelerates research. Its modular design and support for both discrete and continuous environments make it versatile for various applications.
Overview
Clean PyTorch implementations of imitation and reward learning algorithms
Use Cases
Best For
- Training RL agents from expert demonstrations without hand-crafted rewards
- Implementing and benchmarking state-of-the-art imitation learning algorithms
- Research projects requiring reproducible baselines for imitation or reward learning
- Building AI systems that learn from human preferences or comparisons
- Educational purposes to understand imitation learning algorithms in depth
- Developing robotics controllers using demonstration data
Not Ideal For
- Teams using TensorFlow as their primary deep learning framework
- Projects still reliant on the deprecated OpenAI Gym API
- Applications requiring Maximum Causal Entropy IRL or Soft Q Imitation Learning with continuous action spaces
- Developers needing a lightweight, no-dependency setup for rapid prototyping
Pros & Cons
Pros
Implements key algorithms like GAIL, AIRL, and preference-based learning, all documented with benchmark results for reliable performance comparison.
Emphasizes clean, modular, and well-tested code, ensuring stability for research and practical applications as stated in the philosophy.
Features thorough API docs, tutorials, and benchmark summaries on ReadTheDocs, making it accessible for varied use cases.
Offers consistent interfaces across algorithms, simplifying experimentation and integration with gymnasium environments.
Cons
Algorithms like MCE IRL and SQIL do not support continuous action spaces, restricting their utility in environments like robotics simulations.
Only compatible with the newer gymnasium API, forcing migration from the older gym, which can be a barrier for existing projects.
Uses Sacred for configuration, adding overhead for users unfamiliar with it or those preferring simpler script-based workflows.
Frequently Asked Questions
Related Projects
Stable Baselines3PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
GymnasiumAn API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
DopamineDopamine is a research framework for fast prototyping of reinforcement learning algorithms.
TianshouAn elegant PyTorch deep reinforcement learning library.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.