Notes on Score-based Generative Models

Introduction

In 2019, Song et al. ¹ introduced a new family of generative models called score-based generative models. The main idea is that if we can learn the score functions, i.e. the gradients of log probabiltiy density function, then we can generate samples with Langevin-type sampling. The proposed method is able to generate GAN-level samples without adversarial training, which is known to be troublesome in practice.

Score matching

Langevin dynamics

Score-based generative models

Concluding remarks