A new way to use diffusion-based denoisers inside ADMM with provable convergence
This paper shows how to combine modern score-based generative models with a popular optimization algorithm, the alternating direction method
This paper shows how to combine modern score-based generative models with a popular optimization algorithm, the alternating direction method of multipliers (ADMM), in a way that both improves results and comes with convergence guarantees. The authors identify why previous attempts struggled: the score functions are trained on images with Gaussian noise, while the internal variables that ADMM produces do not follow the same noise geometry. They propose a three-stage denoiser, called AC-DC, to correct that mismatch and make score-based denoising effective inside ADMM.
Score-based generative models (also called diffusion models) learn a function called a score that approximates the gradient of the log density of noisy data. Practically, these scores can be used as denoisers via a simple formula (Tweedie’s lemma) or by running a short score-based ordinary differential equation (ODE). ADMM is an optimization method often used to solve inverse problems (for example, recovering an image from blurred or partial measurements). ADMM uses a dual variable that changes the geometry of the intermediate iterates, which makes the iterates look different from the noisy examples the score was trained on. That mismatch can break denoising and make the algorithm unstable.
To fix this, the authors design an AC-DC denoiser to sit in ADMM’s denoising step. The AC stage (auto-correction) adds controlled Gaussian noise to pull the ADMM iterate toward the kinds of noisy examples the score knows. The DC stage (directional correction) runs a short conditional Langevin dynamics process. This step nudges the iterate in directions that align with the noise-trained manifolds while trying not to erase the true signal. The final stage applies the score-based denoiser itself, implemented either by the Tweedie-based closed form or by integrating a score ODE initialized from the DC result. The same denoiser can also be used in other proximal-type algorithms, not only ADMM.