A Python package for training PyTorch neural networks using variational inference for Bayesian deep learning.
PyVarInf is a Python package that facilitates Bayesian deep learning by enabling variational inference for PyTorch neural network models. It allows users to train models that output posterior distributions over weights rather than point estimates, providing uncertainty quantification and enabling Bayesian model averaging. The library transforms standard PyTorch models into variational versions with Gaussian posteriors through a simple API.
Machine learning researchers and practitioners working with PyTorch who want to incorporate Bayesian methods into their deep learning pipelines, particularly those interested in uncertainty estimation, model selection, or information-theoretic approaches to neural network training.
PyVarInf provides a straightforward way to apply variational inference to existing PyTorch models without requiring extensive modifications, offering multiple prior distributions and seamless integration with PyTorch's optimization ecosystem. It bridges the gap between practical deep learning and Bayesian methodology with an accessible implementation.
Python package facilitating the use of Bayesian Deep Learning methods with Variational Inference for PyTorch
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
The Variationalize function transforms standard PyTorch models into variational versions with Gaussian posteriors using a single line of code, as demonstrated in the README example with a CNN.
Supports various priors including Gaussian, conjugate, and mixture of Gaussians, enabling flexible Bayesian modeling for different assumptions, detailed in the prior configuration section.
Seamlessly works with existing PyTorch optimizers and loss functions, allowing training with familiar tools like Adam optimizer without modification.
Implements variational inference with a Minimum Description Length interpretation, linking model complexity to data encoding for principled model selection.
Setting up non-Gaussian priors requires specifying multiple parameters (e.g., n_mc_samples, alpha_0, beta_0), which the README admits can be intricate and user-defined, increasing setup complexity.
Monte Carlo gradient estimation adds sampling overhead, slowing down training compared to deterministic methods due to extra stochastic steps.
Only Gaussian posteriors are supported, restricting variational approximations to this family and potentially limiting flexibility for complex models.