Annealed L\'{e}vy Diffusion via Gaussian Surrogate Reverse Dynamics
Gaussian diffusion models poorly cover extreme regions for heavy-tailed data because their light-tailed corruption kernels yield exponentially sparse tail samples. This paper introduces Annealed Lévy Diffusion (ALD), which specifies time-dependent $\alpha(t)$-stable perturbation marginals via a Gaussian scale-mixture representation, enabling a two-phase schedule that transitions from heavy-tailed exploration to Gaussian refinement. The construction retains conditional-Gaussian score matching with only one added scalar input, making it compatible with standard diffusion training while targeting distributions with polynomial tails.
The marginal-first idea is clean and empirically promising on synthetic heavy-tailed targets, but the paper oversells theoretical coherence: the preferred ALD-posterior sampler changes the forward marginals relative to the training perturbation family, weakening the grounding of its reverse update. Image experiments are also limited to low-resolution datasets with thin statistical reporting.
The conditional-Gaussian reduction is genuinely elegant. By encoding all heavy-tailed behavior in an independent scalar $S_t$, the authors preserve standard $\epsilon$-prediction training with a single extra conditioning term $\log S_t$, which is a practical win over fractional-score alternatives like LIM. The two-phase $\alpha(t)$ schedule also shows clear benefits on globally heavy-tailed 2D benchmarks.
First, ALD-posterior is a sampler for an alternative forward process whose intermediate marginals differ from those used during training; the exact adjacent posterior is only valid under this modified process, yet the paper treats it as a superior drop-in replacement without analyzing the mismatch. Second, the method relies on a clamped scale $S_{\max}$, a piecewise-linear $\alpha(t)$ schedule with manually chosen $\alpha_{\min}$ and $\tau$, and an empirical data mean at $t=1$, all of which are ad-hoc knobs without principled selection rules. Third, there is no convergence or score-estimation theory under annealing; claims of training efficiency rest on a single clipped Holtsmark sweep with small networks.
On 2D synthetic benchmarks, ALD-posterior outperforms DDPM and DLPM on the heavy-tailed Holtsmark target and remains competitive on light-tailed or manifold data, supporting the core claim. However, the image experiments are thin: CelebA64 results come from a single training and sampling seed, with ALD-posterior chosen from a validation-only sampler sweep that baselines did not receive. While ALD-posterior achieves the best FID and KID in that run, DLPM actually wins on precision, density, and coverage, suggesting a real trade-off rather than uniform superiority. The new SHS metric is also unvalidated and defined specifically for this paper.
No code repository or data release is mentioned. Although Algorithms 1–3 lay out the training and sampling loops, reproducing the results would require a nontrivial implementation of the Kanter–Chambers–Mallows–Stuck stable sampler, the generalized ALD-posterior grid schedule, and dataset-specific hyperparameter searches for $\alpha_{\min}$, $\tau$, $S_{\max}$, and the time-grid power $p$ (Appendix A.6). Appendix A does list architectures, learning rates, and EMA settings, but the lack of a reference implementation is a meaningful barrier.
Gaussian diffusion models offer a practical framework for generative modeling. Denoising L\'{e}vy Probabilistic Models (DLPM) introduce heavy-tailed perturbations with a fixed tail index throughout denoising. We introduce Annealed L\'{e}vy Diffusion (ALD). ALD varies the stability index $\alpha(t)$ to combine heavy-tailed exploration with Gaussian refinement. Our marginal-first formulation represents stable noise as a scalar Gaussian scale mixture. This formulation enables standard Gaussian noise-prediction training with one additional scale input. We study two sampling processes. ALD-renoise estimates the clean sample and injects fresh marginal noise. ALD-posterior preserves dependence between successive states through an accumulated conditional variance and a Gaussian bridge update. Both samplers reuse the same trained predictor with different scalar inputs. We evaluate ALD on four 2D benchmarks and an imbalanced nine-mode mixture. We also examine training efficiency and the effects of prediction perturbations on clipped Holtsmark data. Additional experiments evaluate MNIST and CelebA64 generation and CIFAR-10LT augmentation. ALD preserves local geometry on the evaluated light-tailed and manifold-structured targets.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.