Levanter

Uses Haliax for composable, easy-to-follow code that maintains high performance, making complex model training more legible and debuggable.

Ensures reproducible results even after preemption and resumption on TPUs, which is critical for rigorous scientific research and reliable experiments.

Supports TPUs and GPUs with FSDP and tensor parallelism, rivaling commercial frameworks like MosaicML Composer in performance for large models.

Imports and exports models, tokenizers, and datasets via SafeTensors, facilitating seamless use of the HF ecosystem within JAX-based pipelines.

Includes the Sophia optimizer for potentially 2x faster training than Adam, offering state-of-the-art optimization for faster convergence.