DETR

Vision Transformer

This repository provides the official implementation of Vision Transformer (ViT) and MLP-Mixer architectures for image recognition, based on seminal research papers from Google Research. It includes pre-trained models on datasets like ImageNet and ImageNet-21k, along with code for fine-tuning on custom datasets using JAX and Flax. ## Key Features - **Vision Transformer (ViT)** — Applies transformer architecture to image patches for scalable image recognition. - **MLP-Mixer** — An all-MLP architecture for vision tasks, offering an alternative to convolutional networks. - **Pre-trained Models** — Includes a wide variety of ViT and Mixer models (e.g., ViT-B/16, ViT-L/16, Mixer-B/16) pre-trained on ImageNet and ImageNet-21k. - **Fine-tuning Support** — Provides configurable scripts to fine-tune models on datasets like CIFAR-10, CIFAR-100, and custom datasets. - **LiT Models** — Includes Locked-image text Tuning models for zero-shot transfer learning with image-text alignment. - **Cloud Integration** — Supports training on Google Cloud VMs with GPU or TPU accelerators. ## Philosophy The project emphasizes reproducibility and accessibility of state-of-the-art vision models, offering well-documented code and pre-trained checkpoints to facilitate research and practical applications in computer vision.

Big Transfer (BiT)

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.

mip-NeRF

Mip-NeRF is an extension of Neural Radiance Fields (NeRF) that addresses aliasing artifacts by representing scenes at continuously-valued scales. It renders anti-aliased conical frustums instead of single rays, enabling higher-quality synthesis of novel views from 2D images while being faster and more compact than the original NeRF. ## Key Features - **Multiscale Scene Representation** — Models scenes at continuous scales to handle varying image resolutions. - **Anti-Aliased Rendering** — Renders conical frustums instead of rays, reducing blur and aliasing artifacts. - **Improved Detail Preservation** — Significantly enhances NeRF's ability to capture fine details. - **Computational Efficiency** — 7% faster than NeRF and half the model size, while reducing error rates by 17-60%. - **Scalable Performance** — Matches brute-force supersampled NeRF accuracy while being 22x faster on multiscale datasets. ## Philosophy Mip-NeRF is designed to efficiently solve the aliasing problem in neural rendering by integrating multiscale representation directly into the NeRF framework, prioritizing both rendering quality and computational performance.

JAX RL

JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.