Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors
Discriminative models generalize to unseen samples, suggesting they capture data-manifold structure beyond their task labels. This paper asks whether that structure can be exposed as a generative signal without retraining, labels, or auxiliary generators. The authors propose the Discriminative Score Function (DSF), which turns the loss-gradient geometry of a fixed discriminative model into an image-space update rule. DSF enables unconditional generation, conditional guidance, inpainting, and zero-shot quantization calibration across classifiers, detectors, and self-supervised encoders.
DSF offers an elegant theoretical framing—turning discriminative loss-gradient geometry into a generative field via the Loss Tangent Kernel—but its practical impact is split. The zero-shot quantization results are the strongest evidence: "DSF matches or exceeds the real-image oracle in nearly all settings and consistently outperforms data-free calibration sources for a quantized DINOv2 feature extractor." Yet the generation experiments are purely qualitative, and the authors concede that "DSF is viewed as a training-free generative interface induced by a discriminative model, not a replacement for high-fidelity diffusion generators."
The cross-architecture transfer is compelling. "DSF produces model-dependent visual structure in all cases: the classifier tends to generate object-centric patterns, DINO produces broad natural-image structure from its self-supervised representation, and DETR yields scene-like samples with multiple object-like regions." The quantization application is concrete and practically useful: "DSF-generated samples also provide effective zero-shot quantization calibration when real images or labels are unavailable, preserving feature geometry better than data-free calibration baselines."
The generation results rely entirely on qualitative samples without standard metrics such as FID or IS, making fidelity claims impossible to verify numerically. More fundamentally, the method hinges on treating $-\theta$ as a proxy for the aggregate training gradient; while the authors write that "the practical assumption is that the learned parameter direction summarizes a useful trace of training-induced aggregate descent," this is exact only under idealized $\ell_2$-regularized stationarity and becomes tenuous for modern optimizers, learning-rate schedules, and normalization layers. The limitation note that "its gradient proxy can depend on architecture and training recipe" confirms this fragility.
Quantitative evidence is strong only for quantization. The caption of Table 1 notes that "DSF matches or exceeds the real-image oracle in nearly all settings and consistently outperforms data-free calibration sources for a quantized DINOv2 feature extractor," and the fair sweep over RegCache, Gaussian noise, and MimiQ supports this. For generation, however, there are no numerical comparisons against prior data-free synthesis methods such as DeepInversion; the paper differentiates itself conceptually by noting that "existing discriminative-to-generative methods typically rely on external or task-specific handles," but the fidelity gap remains unmeasured.
Algorithm 1 and Appendix C specify the update, layer subsets, and schedules, yet no public code repository is mentioned in the main text. Reproduction is further constrained by compute: "Because Step 3 in Algorithm 1 preserves the parameter-gradient graph, DSF requires mixed second-order differentiation: the first derivative forms $g_{\ell}=\nabla_{\theta_{\ell}}\ell_{aug}$, and the image update differentiates the alignment objective through $g_{\ell}$. In A6000 sweeps with batch size 50, this costs about 1.4–1.7 seconds per image." The activation-scale multipliers used in quantization (Appendix, Table 4) were selected by per-source sweeps, adding another hyperparameter surface. An independent reproduction is theoretically possible but blocked by high per-sample cost and the absence of released code.
Discriminative models are trained with task-specific objectives such as classification, detection, or representation learning, yet their practical utility depends on generalizing beyond the training task to unseen samples from the same domain. This suggests that a discriminative model may retain an implicit trace of the data manifold beyond its task labels. We ask whether this trace can be exposed as a generative signal without task-specific guidance. We introduce the Discriminative Score Function (DSF), which extracts an image-space generative field from the loss-gradient geometry of a pretrained discriminative model. DSF provides a training-free update rule that moves images toward model-consistent visual regions without retraining or task-specific targets. We apply DSF to diverse pretrained discriminative models, including ResNet classifiers, DETR detectors, and DINOv2 ViT encoders. DSF generates unconditional images from noise and supports conditional guidance, editing, inpainting, and explanation through the same field. DSF-generated samples also provide effective zero-shot quantization calibration when real images or labels are unavailable, preserving feature geometry better than data-free calibration baselines. These results show that discriminative training leaves a generative functional signal for synthesis and calibration. Project page: https://junhoo.me/dsf.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.