Parameter Expansion for Data Augmentation

Generative Modeling Explained

Ying Nian Wu's UCLA Statistical Machine Learning Tutorial on generative modeling.

Latent Space Factorisation and Manipulation via Matrix Subspace Projection

ICML'20, 2020. [All Versions]

Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling

International Journal of Computer Vision, 1998. [All Versions]. [Preprint]. This article presents a statistical theory for texture modeling. This theory combines filtering theory and Markov random field modeling through the maximum entropy principle, and interprets and clarifies many previous concepts and methods for texture analysis and synthesis from a unified point of view. The theory characterizes the ensemble of images I with the same texture appearance by a probability distribution f(I) on a random field, and the objective of texture modeling is to make inference about f(I), given a set of observed texture examples

Cooperative Training of Descriptor and Generator Networks

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018. [All Versions]. This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. This work observes that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling