Are We Overthinking One-Step Generation?

cs.LG Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung Han · Oct 14, 2026

What it does

Why it matters

This paper introduces Priming Models, a two-stage framework that first pretrains with a simple reconstruction objective—surprisingly, even fixed-noise denoising suffices—and then refines the network with feature-space distributional...

Main concern

Community signal

0 up · 0 down

AI Review AI reviewed

Plain-language introduction

One-step generative models typically demand complex distillation or fragile end-to-end distributional training. This paper introduces Priming Models, a two-stage framework that first pretrains with a simple reconstruction objective—surprisingly, even fixed-noise denoising suffices—and then refines the network with feature-space distributional losses. The core claim is that this reconstruction stage fully removes the training difficulty of distributional optimization, enabling state-of-the-art quality at roughly one-tenth the compute of prior one-step methods.

Critical review

Verdict

Bottom line

Priming Models deliver a convincingly strong empirical efficiency–quality trade-off on ImageNet, text-to-image, and world-modeling benchmarks. The paper’s central thesis—that reconstruction pretraining alone can eliminate the need for elaborate stabilizers like style embeddings or classifier-free guidance—is well supported by compute-matched comparisons and ablations. However, the theoretical grounding remains thin: the Jacobian-alignment argument in Section 2.2 is suggestive but based on a small, correlational experiment over eight classes, and the method still depends heavily on pretrained feature encoders whose role is not fully characterized.

“Our central scientific finding is that this simple reconstruction stage is already enough to entirely alleviate the difficulty of distributional training.”

Park et al., Abstract · Abstract

“Their training recipes nevertheless rely on longer training-time schedules and subtle tricks like style-conditioning... Thus, the simplicity of one-step inference still comes at the cost of substantial complexity in getting these models to train.”

Park et al., Sec. 1 · Section 1

What holds up

The efficiency gains are large and consistent. The ImageNet results are the strongest among methods evaluated under a common protocol, and the ablation in Section 4.1 robustly supports the fixed-noise reconstruction intuition. The fact that denoising at a single corruption level $s=0.8$ outperforms a broad range of alternatives, including masking and RAE distillation, reinforces the simplicity of the recipe.

“On ImageNet at 256×256 resolution, we achieve a state-of-the-art FID of 1.19 among one-step generators using approximately one-tenth the training compute of prior methods with comparable FID.”

Park et al., Abstract/Table 1 · Abstract

“After 12,500 OT post-training updates, s=0.8 yields the lowest terminal EMA FID (1.65), followed by s=0.7 (1.76), s=0.9 (1.97), s=0.5 (2.46), and s=1.0 (65.08).”

Park et al., Sec. 4.1 · Section 4.1

Main concerns

The paper’s primary weakness is its reliance on off-the-shelf pretrained encoders for post-training, a dependency the authors acknowledge but do not resolve. In addition, the exclusion of FD-loss pixel MeanFlow (FID 0.72) from the main comparison slightly blurs the pixel-space SOTA claim. The Jacobian alignment analysis, while intuitive, is limited in scale and does not establish causality; it measures correlation on eight classes and relies on a hand-chosen PCA truncation $r=64$.

“Priming Models rely on pretrained feature encoders. Although we demonstrate our method with multiple sets of feature encoders, this dependence raises the question of whether it can be eliminated.”

Park et al., Sec. 5 · Section 5

“FD-loss (Yang et al., 2026) reports FID 0.72 for pixel MeanFlow by directly optimizing Fréchet Distance. We exclude this configuration from comparison and revisit this in Section 4.2.”

Park et al., Footnote 1 · Footnote 1

“Figure 4 reports the average over eight classes.”

Park et al., Sec. 2.2 · Section 2.2

Evidence and comparison

The evidence is generally solid: training compute is carefully reported in EFLOPs inclusive of both stages and feature encoders, and the held-out FDr$^5$ metric across five non-training feature spaces strengthens the FID claims. That said, comparisons in text-to-image and world modeling use different base architectures than some baselines, and the perceptual loss used during pretraining itself requires a frozen ConvNeXt encoder, meaning the method is not entirely self-contained.

“For Priming Models, this accounts for both pretraining and distributional post-training, including feature-encoder computations.”

Park et al., Sec. 3.1 · Section 3.1

“These five evaluation encoders are not used in the training of either Priming Models or the compared baselines.”

Park et al., Sec. 3.1 · Section 3.1

Reproducibility

Reproducibility is mixed. The authors open-source their code and provide detailed hyperparameters in Appendix C, which is excellent. Yet independent reproduction would require replicating several underspecified design choices: the pretrained ResNet-MAE and ConvNeXt feature branches and their calibrated fixed scales, the branch weights ($1/2$ each), the specific Sinkhorn regularization schedule, and the perceptual loss coefficients ($\lambda=0.4$ for LPIPS, $\lambda=0.1$ for ConvNeXt). These are not negligible barriers.

“We open-source our codebase at https://github.com/JaeyeonKim01/priming.”

Park et al., Abstract · Abstract

“We follow the feature choice setup of (Deng et al., 2026), using the 84 feature outputs from the frozen ResNet-MAE encoder and 13 from ConvNeXt.”

Park et al., Appendix C.1 · Appendix C.1

“Before post-training, for each feature space, we calibrate a fixed scale σ using the median pairwise Euclidean distance among 2,000 training images sampled once with a fixed seed and divide each feature value by this σ.”

Park et al., Appendix C.1 · Appendix C.1

Abstract

Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This has motivated extensive research on one-step generative modeling primarily built on diffusion and flow, though existing approaches often rely on complex training machinery. In this work, we introduce Priming Models, a simple two-stage framework for training one-step generators. Motivated by the difficulty of distributional training from scratch, Priming Models are first trained on a reconstruction objective, e.g., pixel-space denoising at a fixed noise level, before undergoing feature-space distributional post-training. Our central scientific finding is that this simple reconstruction stage is already enough to entirely alleviate the difficulty of distributional training. Empirically, across class-conditional image generation, text-to-image generation, and world modeling, Priming Models significantly advance the Pareto frontier of training compute vs. generation quality. On ImageNet at $256 \times 256$ resolution, we achieve a state-of-the-art FID of $1.19$ among one-step generators using approximately one-tenth the training compute of prior methods with comparable FID. We open-source our codebase at https://github.com/JaeyeonKim01/priming.

Challenge the Review

Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.

Challenges are public to read, but only signed-in members can post them. Your challenge text is stored with your account for moderation, but usernames are not shown in the public thread.

No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.