Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion
Meta-Diff reframes few-shot diffusion adaptation as reusable transport in a low-dimensional task space, avoiding per-target backbone retraining. A shared encoder maps source and target distributions to coordinates $z$, and a learned affine map $\widetilde{z}_T = A z_S + b_c$ predicts the target coordinate from a data-rich source without using any target images at deployment. This is especially consequential in domains like healthcare, where target data are scarce and collecting them is costly or privacy-sensitive.
The paper presents an elegant, lightweight framework and an unusually honest empirical diagnosis. Distributional gains on CIFAR-100 and controlled GMMs are real in low-data regimes, but the practical value for natural image shifts is undermined by a simple image-space corruption baseline that outperforms task-space transport on CIFAR, and by severe semantic failures on Office-Home where object identity collapses despite domain alignment.
The 210-run GMM factorial study is a standout contribution that isolates necessary success conditions—usable task identity, relation predictability, transport preservation, and headroom over source reuse—under known ground truth. The authors are also commendably self-critical, openly stating that their theory guarantees only coordinate concentration and stability, not relational generalisation or semantic sufficiency, and they rigorously validate source specificity via paired wrong-source controls on CIFAR.
The most damaging empirical result is that a trivial image-space baseline defeats the learned transport on CIFAR: generating from the clean source and then applying the known corruption achieves lower SW than task-space transport. On Office-Home, requested-object identity is only 1.67%, and even bypassing transport with abundant direct target coordinates yields below 2.5%, indicating that the coordinate representation or the generator itself cannot preserve semantics under natural domain shifts. Additionally, the target-budget comparison against full fine-tuning is confounded because optimisation steps and learning rates vary across budgets, preventing a clean isolation of data quantity.
The GMM evidence strongly supports the operating-regime claims and the comparative framing against target-only adaptation is fair, but the CIFAR adaptation comparison does not isolate target-data quantity since steps and learning rates differ across budgets. Conditioning mechanism comparisons are also inconclusive: the matched conditioning SW intervals contain zero at all three target budgets, so no reliable FiLM–basis ranking is established.
Experimental specifications are exceptionally thorough: exact loss expressions, encoder architectures, transport maps, bootstrap resampling schemes, data splits, and adaptation hyperparameters are all documented. However, no public code repository is mentioned in the text, and some architecture ablations rely on a single training seed, leaving seed variation unquantified.
Many deployments require a model to move rapidly from a data-rich source to a related target observed only sparsely. This is especially consequential where target data are difficult to collect, as in healthcare settings with persistent population or acquisition imbalances. Retraining or fine-tuning a high-capacity generator for every target is costly and can be statistically unreliable when target data are scarce. We introduce Meta-Diff, which instead learns a recurring source--target relation once in a compact task space. A shared encoder maps each distribution to a low-dimensional coordinate; a learned relation transports a new source coordinate to its predicted target, and a shared diffusion backbone generates from that prediction without target-specific retraining. Controlled Gaussian-mixture experiments identify when this works: task identity must survive the representation, the relation must generalise across tasks, and the source--target shift must leave useful headroom for transport. On held-out CIFAR-100 validation tasks, correct source $+$ transport achieves 13.9% lower mean SW than transport from the wrong source. With validation-selected adaptation settings in the final-test one-target comparison, its mean SW is 14.7% lower than target-only modelling and 32.7% lower than full fine-tuning; the main route itself uses no target image and assumes the applicable relation is known. On matched CIFAR test episodes, generating from the clean-source coordinate and then applying the known corruption yields lower SW than task-space transport. Office-Home exposes a complementary boundary: transport can align target-domain statistics without reliable object identity. Together, these results recast extreme-few-shot adaptation as reusable transport in task space, while making the conditions for its success and failure directly measurable.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.