A high-level neural network API for specifying and analyzing infinite-width neural networks as Gaussian Processes in Python.
Neural Tangents is a Python library for working with infinite-width neural networks, treating them as Gaussian Processes with computable kernels. It allows researchers to analyze neural network architectures in their infinite limit, providing tools to compute the Neural Tangent Kernel (NTK) and Neural Network Gaussian Process (NNGP) kernel directly from network specifications.
Machine learning researchers and practitioners studying neural network theory, kernel methods, or training dynamics, particularly those interested in the infinite-width limit of deep learning models.
It uniquely combines ease of use with theoretical rigor, offering a high-level API compatible with JAX to compute infinite network kernels analytically, enabling fast experimentation without sacrificing performance or scalability.
Fast and Easy Infinite Neural Networks in Python
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Leverages JAX and XLA for optimized execution on GPU/TPU with near-perfect multi-accelerator scaling, as shown in benchmark tables for networks without pooling.
Acts as a drop-in replacement for JAX's stax API, extending it with a kernel_fn for easy infinite network analysis while maintaining familiar syntax.
Provides closed-form Gaussian Process inference for infinite networks trained with MSE loss, enabling analytical training dynamics without simulation.
Includes linearize and empirical_kernel_fn to study wide but finite networks, bridging theoretical infinite-width results with practical deep learning.
The README admits low core utilization on CPU and excessive padding on TPU for CNNs with pooling, explicitly recommending NVIDIA GPUs for better performance.
Compared to JAX's stax, nt.stax has different layers and options (e.g., no BatchNorm, but adds LayerNorm), which may restrict architecture design for some users.
Primarily targets theoretical ML research on infinite networks, making it less practical for applied projects that don't require kernel methods or linearization tools.
Requires proper JAX installation with GPU/TPU support, which can be non-trivial and adds setup overhead compared to more standalone deep learning libraries.