The joint work of Chongxuan Li (tenure-track assistant professor in Gaoling School of AI, Renmin University of China), Prof. Bo Zhang (Tsinghua University) and Prof. Jun Zhu (Tsinghua University), titled “Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models” won the Outstanding Paper Award of the international conference on learning representations (ICLR) 2022.

There are 7 papers out of 3391 valid submissions chosen as recipients of the Outstanding Paper Award, due to their excellent clarity, insight, creativity, and potential for lasting impact.

VALUE (Official Comments by ICLR):

Diffusion probabilistic model (DPM), a class of powerful generative models, is a rapidly growing topic in machine learning. This paper aims to tackle the inherent limitation of the DPM models, which is the slow and expensive computation of the optimal reverse variance in DPMs. The authors first present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms with respect to its score function. Then they propose Analytic-DPM, a novel and elegant training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. This paper is significant both in terms of its theoretical contribution (showing that both the optimal reverse variance and KL divergence of a DPM have analytic forms) and its practical benefit (presenting a training-free inference applicable to various DPM models), and will likely influence future research on DPMs.

A short bio for Chongxuan Li

Chongxuan Li received his Ph.D. in the department of computer science and technology at Tsinghua University in 2019. His research interest is mainly in probabilistic machine learning, especially deep generative models, approximate inference, and probabilistic methods for semi-supervised learning and continual learning. His work has been published in top-tier ML journals and conferences, including IEEE Trans. PAMI, ICML, NeurIPS, ICLR, etc. His work Triple-GAN is the optimal GAN (w.r.t. consistency) for semi-supervised learning and was cited over 375 times according to Google Scholar; His work Analytic-DPM is the optimal DPM (w.r.t. likelihood evaluation) and won the Outstanding Paper Award at ICLR 2022. He won MSRA fellowship in 2017, the CCF Distinguished Ph.D. Dissertation Award in 2019, the Chinese Postdoctoral Innovative Talent Support Program in 2019, and Wu Wenjun first Prize of Artificial Intelligence natural Science in 2021. His research was supported by NSFC (General Program).