How to install JAXRL with GPU support on Ubuntu?

Follow the README's prerequisites: install Poetry and MuJoCo dependencies via apt-get, then use pip for JAX CUDA packages. Set environment variables for LD_LIBRARY_PATH as detailed.

JAXRL vs Stable Baselines3 for continuous control?

JAXRL offers JAX-based, research-focused implementations for prototyping, while Stable Baselines3 provides production-ready, PyTorch baselines with broader algorithm support. Choose JAXRL for JAX experiments and extensions, Stable Baselines3 for benchmarking.

Can I use JAXRL for offline reinforcement learning?

Yes, it includes AWAC, an algorithm designed for offline RL, but note that it's implemented for continuous control tasks and may require adjustments for other environments.

How to extend JAXRL for custom environments?

Leverage the modular code structure: modify the environment wrappers in examples and adapt the training loops. The clean design makes it straightforward to integrate new Gym-compatible environments.

Is JAXRL suitable for pixel-based tasks?

Yes, it implements DrQ with image augmentation for pixel-based continuous control, but only supports K=1, M=1 augmentation as noted in the README, which might limit some use cases.

What hardware is needed to run JAXRL efficiently?

A GPU with CUDA support is recommended for performance; the README includes tips for GPU memory allocation and EGL rendering for remote machines to handle video saving.

Should I use JAXRL or jaxrl2 for new projects?

The README links to jaxrl2 as an updated version, so for new work, check jaxrl2 for latest features and maintenance, while JAXRL serves as a reference for specific implementations.

JAX RL

MITJupyter Notebook

JAX (Flax) implementations of reinforcement learning algorithms for continuous action spaces, designed for research.

GitHub

What is JAX RL?

JAXRL is a collection of reinforcement learning algorithms implemented in JAX and Flax, specifically designed for continuous action space environments. It provides clean, research-focused implementations of algorithms like SAC, AWAC, DDPG, and REDQ to help researchers build upon and experiment with modern RL techniques.

Target Audience

Reinforcement learning researchers and practitioners who want to experiment with JAX-based implementations of continuous control algorithms, particularly those working on offline/online RL, pixel-based control, or algorithmic extensions.

Value Proposition

It offers simple, modular implementations optimized for JAX's performance benefits (GPU acceleration, automatic differentiation) while maintaining readability for research prototyping, unlike more complex baseline repositories.

Overview

JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.

Use Cases

Best For

Research prototyping of new RL algorithms in continuous action spaces

Related Projects

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub

757 stars75 forks0 contributors

Experiments with JAX and Flax for reinforcement learning

Implementing and extending off-policy actor-critic methods

Pixel-based continuous control with data augmentation

Fast model-free RL with ensemble methods like REDQ

Educational understanding of modern RL algorithm implementations

Not Ideal For

Production benchmarking or baseline comparisons where optimized, stable implementations are needed
Research involving discrete action spaces or environments outside continuous control
Teams requiring a comprehensive RL library with extensive algorithm coverage beyond the few implemented

Pros & Cons

Pros

Clean Research Code

Implementations are simple and modular, prioritizing readability for easy extension and experimentation, as stated in the repository's goal.

JAX Performance Benefits

Leverages JAX for GPU acceleration and automatic differentiation, with installation notes for CUDA support enabling fast training on hardware.

Modern Algorithm Suite

Includes key algorithms like SAC with learnable temperature and AWAC for offline RL, supported by citations and example results for continuous control.

Research-Focused Design

Explicitly aimed at prototyping and extensions rather than benchmarking, making it ideal for algorithmic modifications and new experiments.

Cons

Complex Setup Process

Installation requires Poetry, specific system dependencies for MuJoCo, and manual GPU configuration, which can be error-prone and time-consuming.

Limited Maintenance

The README points to an updated version (jaxrl2), suggesting this repository may be outdated or less actively supported for new features.

Niche Use Case

Focus on continuous control and specific algorithms means it lacks support for discrete actions or broader RL tasks, limiting its applicability.

Frequently Asked Questions

Home

JAX

Vision Transformer

This repository provides the official implementation of Vision Transformer (ViT) and MLP-Mixer architectures for image recognition, based on seminal research papers from Google Research. It includes pre-trained models on datasets like ImageNet and ImageNet-21k, along with code for fine-tuning on custom datasets using JAX and Flax. ## Key Features - **Vision Transformer (ViT)** — Applies transformer architecture to image patches for scalable image recognition. - **MLP-Mixer** — An all-MLP architecture for vision tasks, offering an alternative to convolutional networks. - **Pre-trained Models** — Includes a wide variety of ViT and Mixer models (e.g., ViT-B/16, ViT-L/16, Mixer-B/16) pre-trained on ImageNet and ImageNet-21k. - **Fine-tuning Support** — Provides configurable scripts to fine-tune models on datasets like CIFAR-10, CIFAR-100, and custom datasets. - **LiT Models** — Includes Locked-image text Tuning models for zero-shot transfer learning with image-text alignment. - **Cloud Integration** — Supports training on Google Cloud VMs with GPU or TPU accelerators. ## Philosophy The project emphasizes reproducibility and accessibility of state-of-the-art vision models, offering well-documented code and pre-trained checkpoints to facilitate research and practical applications in computer vision.

Stars12,547

Forks1,468

Last commit2 months ago

Big Transfer (BiT)

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.

Stars1,541

Forks175

Last commit1 year ago

mip-NeRF

Mip-NeRF is an extension of Neural Radiance Fields (NeRF) that addresses aliasing artifacts by representing scenes at continuously-valued scales. It renders anti-aliased conical frustums instead of single rays, enabling higher-quality synthesis of novel views from 2D images while being faster and more compact than the original NeRF. ## Key Features - **Multiscale Scene Representation** — Models scenes at continuous scales to handle varying image resolutions. - **Anti-Aliased Rendering** — Renders conical frustums instead of rays, reducing blur and aliasing artifacts. - **Improved Detail Preservation** — Significantly enhances NeRF's ability to capture fine details. - **Computational Efficiency** — 7% faster than NeRF and half the model size, while reducing error rates by 17-60%. - **Scalable Performance** — Matches brute-force supersampled NeRF accuracy while being 22x faster on multiscale datasets. ## Philosophy Mip-NeRF is designed to efficiently solve the aliasing problem in neural rendering by integrating multiscale representation directly into the NeRF framework, prioritizing both rendering quality and computational performance.

Stars939

Forks112

Last commit3 years ago

NesT

Nested Hierarchical Transformer https://arxiv.org/pdf/2105.12723.pdf

Stars204

Forks27

Last commit2 months ago