A JAX-native library implementing Monte Carlo tree search algorithms like AlphaZero and MuZero for reinforcement learning research.
Mctx is a JAX-native library implementing Monte Carlo tree search (MCTS) algorithms like AlphaZero, MuZero, and Gumbel MuZero. It enables efficient search-based reinforcement learning by combining learned neural network models with parallelizable, JIT-compiled search algorithms that operate on batches of inputs.
Reinforcement learning researchers and practitioners working on search-based agents who need efficient, hackable implementations of MCTS algorithms that integrate with JAX-based neural network models.
Developers choose Mctx because it provides JAX-native implementations that fully support JIT compilation for hardware acceleration while maintaining Python usability, striking a balance between performance and hackability for research exploration. It includes batch-parallel search algorithms designed to work with large learned environment models.
Monte Carlo tree search in JAX
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Fully supports JIT compilation for hardware acceleration, enabling fast computations on GPUs and TPUs as highlighted in the README.
Operates on batches of inputs in parallel, maximizing accelerator utilization for efficient search-based reinforcement learning.
Highly configurable search algorithms allow extensive experimentation, supporting research exploration in search-based agents per the library's philosophy.
Designed to work with deep neural networks for environment models, facilitating advanced algorithms like MuZero and Gumbel MuZero.
Includes a policy that guarantees improvement given correctly evaluated action values, a key feature for reliable agent training.
Requires users to define custom RootFnOutput and recurrent_fn functions, which involves significant setup and deep learning expertise, adding overhead.
Tightly coupled with JAX, making it incompatible with projects using other frameworks like PyTorch or TensorFlow, limiting flexibility.
Assumes familiarity with both MCTS algorithms and JAX programming, which can be a barrier for newcomers to search-based reinforcement learning.