JAX

JAX allows arbitrary composition of grad, jit, and vmap, enabling complex numerical pipelines without code rewrites, as demonstrated in the per-example gradient example combining all three.

It compiles pure functions with jit using XLA for efficient execution on TPUs, GPUs, and other accelerators, with benchmarks showing significant speedups over plain NumPy.

Supports reverse-mode and forward-mode differentiation through native Python control flow like loops and branches, enabling gradients of derivatives to any order.

Offers compiler-based automatic parallelization, explicit sharding, and manual per-device programming to scale across thousands of devices, as detailed in the scaling section.