Go to AISTATS 2025 Conference homepageMean-Field Microcanonical Gradient Descent
TL;DR: We show that an established sampling method for energy-based models easily overfits, and suggest a remedy by generating several samples simultaneously and make sure they are not too similar.
Abstract: Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white noise, to a low-energy region using gradient descent. We put this model in the framework of normalizing flows, showing how it can often overfit by losing an unnecessary amount of entropy in the descent. As a remedy, we propose a mean-field microcanonical gradient descent that samples several weakly coupled data points simultaneously, allowing for better control of the entropy loss while paying little in terms of likelihood fit. We study these models in the context of stationary time series and 2D textures.
Submission Number: 2097
Loading