Vision Transformer
Official JAX/Flax implementation of Vision Transformer (ViT) and MLP-Mixer for image recognition, with pre-trained models.
What is Vision Transformer?
Vision Transformer is an open-source implementation of transformer-based models for image recognition, including ViT and MLP-Mixer architectures. It provides pre-trained models and tools for fine-tuning on various vision tasks, enabling scalable and efficient image classification. The project also includes LiT models for zero-shot transfer learning with image-text alignment.
Machine learning researchers and practitioners working on computer vision tasks, especially those interested in transformer architectures, model fine-tuning, and reproducible experiments with state-of-the-art vision models.
Developers choose this project for its official implementation of groundbreaking vision models, extensive collection of pre-trained checkpoints, and seamless integration with JAX/Flax for high-performance training on GPUs and TPUs.
Overview
This repository provides the official implementation of Vision Transformer (ViT) and MLP-Mixer architectures for image recognition, based on seminal research papers from Google Research. It includes pre-trained models on datasets like ImageNet and ImageNet-21k, along with code for fine-tuning on custom datasets using JAX and Flax.
Key Features
- Vision Transformer (ViT) — Applies transformer architecture to image patches for scalable image recognition.
- MLP-Mixer — An all-MLP architecture for vision tasks, offering an alternative to convolutional networks.
- Pre-trained Models — Includes a wide variety of ViT and Mixer models (e.g., ViT-B/16, ViT-L/16, Mixer-B/16) pre-trained on ImageNet and ImageNet-21k.
- Fine-tuning Support — Provides configurable scripts to fine-tune models on datasets like CIFAR-10, CIFAR-100, and custom datasets.
- LiT Models — Includes Locked-image text Tuning models for zero-shot transfer learning with image-text alignment.
- Cloud Integration — Supports training on Google Cloud VMs with GPU or TPU accelerators.
Philosophy
The project emphasizes reproducibility and accessibility of state-of-the-art vision models, offering well-documented code and pre-trained checkpoints to facilitate research and practical applications in computer vision.
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.