jax

JAX can differentiate through complex Python structures like loops, branches, and closures to any order, as demonstrated with grad examples for tanh and abs_val functions.

Uses XLA to jit-compile functions for efficient execution on TPUs and GPUs, with the README showing performance comparisons for slow_f vs fast_f.

Transformations like grad, jit, and vmap can be arbitrarily combined, enabling complex pipelines such as per-example gradients with jit(vmap(grad)).

Supports scaling across thousands of devices via automatic or explicit sharding modes, illustrated with code for FSDP and batch parallelism.