SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing
Dataset-use auditing for text-to-image models is difficult because fine-tuned models absorb collection-level visual style without reproducing individual training images. SpectralCanary addresses this by blending a fixed canary image into the low-frequency Fourier content of every collection image before release, preserving captions and avoiding auxiliary generation or per-image optimization. A lightweight verifier then detects the transferred spectral signature in a suspect model's black-box outputs. The method achieves strong detection rates across multiple architectures and domains with minimal fidelity loss, establishing low-frequency traces as a practical auditing mechanism.
The paper makes a compelling empirical case that low-frequency spectral tampering is an effective carrier for collection-level auditing traces in T2I fine-tuning. Its strongest contribution is the controlled, matched-energy ablation showing low-frequency placement outperforms mid/high-frequency placement, combined with unusually broad cross-domain validation. However, the security model is fundamentally secrecy-dependent: disclosure of the canary spectrum enables exact inversion via Eq. 6, and the evaluated adaptive attacks are limited to relatively naive donor replacement. The work is best read as establishing a strong, well-characterized empirical operating point rather than a cryptographically robust guarantee.
The low-frequency design hypothesis is carefully validated by matched-energy ablations on Pokemon and ROCOv2, where the low-frequency band exceeds the high-frequency band by 16.9 and 88.9 percentage points in per-generation TPR at 1% FPR, respectively. The evaluation is broad for this space, spanning five visual domains and two architecture families with near-zero mean generative degradation ($\Delta$FID 0.05) and high image fidelity (SSIM 0.989, LPIPS 0.022). Operationally, the method is attractive because it relies on a single deterministic FFT-based blend that runs in 0.39 seconds per image on CPU without iterative optimization or auxiliary models.
The auditing guarantee is conditional on keeping the canary image and verifier private; secrecy is the only barrier to removal, as exact spectral inversion is possible given $V$, $\alpha$, and $c$. The main experiments also rely on a single natural canary image, leaving open questions about key reuse and independent canary selection. Full-parameter fine-tuning (Appendix G) reveals material degradation under training-time augmentation, with TPR@1% FPR falling to 61.9%, suggesting the headline robustness may be partially specific to LoRA. Finally, the evaluation does not cover realistic deployment scenarios such as mixing a protected collection into a larger unprotected corpus, which could dilute the shared spectral mean shift.
The central claim—that low-frequency traces transfer more reliably—is supported by a well-designed ablation equalizing both the number of modified Fourier coefficients and the total Fourier-domain perturbation energy across bands. Baseline comparisons follow published protocols, but the chosen trade-offs inherently favor SpectralCanary: EnTruth depends on caption triggers that recaptioning removes, DIAGNOSIS relies on a hard trigger token, and SIREN requires costly learned encoder optimization. The generative utility evidence relies on $\Delta$FID, which is near zero on average but varies across datasets and is difficult to interpret as strictly harmless because negative values suggest regularization rather than genuine parity.
Reproduction is feasible because the authors publish a code repository and specify protection hyperparameters ($\delta=1$, $\alpha=0.02$, $c=0.20$), verifier architecture (ResNet-50), and training settings (AdamW, lr $10^{-4}$, 20 epochs). The protection transform itself is fully deterministic given these parameters. However, independent reproduction would require the exact canary image and train/validation index splits, which are tied to the released artifacts. The paper reports three independent runs but does not state the random seeds used for model training, and the CPU/GPU timing comparisons conflate algorithmic differences with implementation-specific overheads across the baselines.
Auditing whether a text-to-image (T2I) model was fine-tuned on a particular image collection is difficult: a model can absorb collection-level visual characteristics without reproducing training images. Proactive dataset-use auditing plants a shared signal in a dataset before release, then later checks whether that signal appears in a suspect model's generated outputs. Existing approaches often trade off verifiability, fidelity, robustness, and deployability. We introduce $\textbf{SpectralCanary}$, which uses a smooth low-pass mask to blend a fixed canary image into every collection image while leaving captions unchanged and requiring neither auxiliary generation nor per-image optimization. A lightweight verifier detects the transferred signal in suspect-model outputs. Across eight tests on the Pokemon dataset and three realistic data transformations, spanning two architectures families, SpectralCanary achieves the highest average per-generation TPR@$1\%$ FPR ($91.1\%$) and fixed threshold model-level accuracy using 30 generations ($94.8\%$) among the evaluated methods. Across five datasets spanning artistic, medical, satellite, and fashion domains with SD3.5, SpectralCanary averages $92.4\%$ per-generation TPR@$1\%$ FPR while preserving the released images (SSIM $0.989$, LPIPS $0.022$) with a near-zero mean change FID (mean $\Delta$FID $0.05$) across datasets. Matched-energy ablations on Pokemon and ROCOv2 support our design choice, low-frequency placement transfers more reliably compared to mid/high-frequency placement. Under the evaluated adaptive attacks, the most effective removal requires aggressive modification and incurs more than $25$ $\Delta$FID points. These results identify low-frequency visual signals as effective carriers for collection-level T2I audit traces.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.