Zero-Shot Domain Generalization via Domain Prompt Learning
The paper tackles Zero-Shot Domain Generalization (ZSDG), where a model must recognize unseen classes in unseen target domains without access to any source-domain images during training. The proposed Domain Prompt Learning (DoP) built on CLIP introduces a two-stage pipeline: Text Domain Learning (TDL) synthesizes diverse style tokens via text-space orthogonality and consistency losses, and Visual Domain Learning (VDL) refines the best-matching token per test image using self-supervised entropy minimization at test time. This matters because it removes the costly and often infeasible requirement to collect and annotate source-domain data, making domain generalization practical for new or scarce classes.
DoP is a pragmatic and conceptually clean extension of prompt learning to the source-free DG setting. The two-stage design yields consistent gains over zero-shot baselines and frequently beats methods trained with source data. However, the absolute improvements over the strongest zero-shot competitor PromptStyler are modest (0.2–1.2 points), and the reliance on per-image test-time optimization raises deployment questions. The authors themselves concede that performance on challenging datasets remains far from saturated.
The motivation for source-free DG is well-articulated, and the TDL+VDL decomposition is intuitive: TDL provides a compact, diverse text-space candidate set without target labels, while VDL adapts only the selected style token $S^*$ at test time, keeping the CLIP encoders frozen. The efficiency argument is concrete: refining a single token cuts latency to 170.5 ms per image, a 63.2% reduction versus TPT. Qualitative Stable Diffusion renderings offer convincing evidence that the learned prompts capture interpretable style concepts.
First, the TDL objective relies on orthogonality as a data-independent surrogate for domain diversity; the authors explicitly note that it is 'not an assumption about real-world domain distributions,' which limits theoretical assurance that the generated styles will cover arbitrary target domains. Second, gains over PromptStyler are marginal—on ResNet-50 the average improvement is only 0.2 points (74.7% vs. 74.9%), and on ViT-L/14 DoP underperforms TPT (83.3% vs. 84.1%). Third, hyperparameters vary sharply across benchmarks ($K=2$ for VLCS but $K=50$ for OfficeHome), suggesting non-trivial dataset-specific tuning without a principled selection rule.
The evidence supports the claim that DoP improves over zero-shot CLIP and TPT on most benchmarks, and the comparisons are generally fair: source-data and source-free methods are clearly marked with ✔/✗ in Tables 2–4. The ablation in Table 6 shows that removing either TDL or VDL drops average accuracy substantially ($75.8\%$ and $77.1\%$ vs. $82.0\%$), supporting the claim that both stages are necessary. A caveat is that the ImageNet evaluation uses ImageNet class names to construct prompts, so the task is effectively zero-shot transfer rather than strict zero-shot recognition; comparisons to CuPL and TPT are appropriate, but the gap to source-data methods remains.
The authors provide a public code link and list standard datasets, which aids reproduction. Key training details are reported: AdamW with learning rate $5\times 10^{-3}$, 10 epochs for TDL, and 2 optimization rounds for VDL. Yet reproducibility could be hampered by underspecified initialization—style tokens are 'initialized from randomly sampled style words,' but the exact vocabulary and sampling procedure are not disclosed. Moreover, VDL depends on dataset-specific thresholds ($\rho$ and $K$) that are tuned empirically per benchmark, so reproducing results on a new domain requires a validation sweep.
Domain generalization (DG) aims to improve the generalization of the model to any unseen domains. Targeting this goal, existing DG methods mainly focus on learning domain-invariant features or prompts from the given source domains. However, these methods rely on source-domain data, and collecting and annotating large-scale single- or multi-source datasets is often infeasible, limiting the practical applicability of DG. To tackle this challenge, we propose {\underline{Do}}main \underline{P}rompt Learning (DoP), a method built on a large vision-language model (e.g., CLIP) for zero-shot domain generalization, which recognizes unseen classes in unseen target domains without access to source-domain data. DoP essentially learns the target domain knowledge via prompt both in text and visual modalities to overcome the domain shift in text-visual modalities under the test-time adaptation formulation. Specifically, DoP first takes text domain learning (TDL) to simulate various distribution shifts by learning diverse style tokens implicitly in the text space with inter-style diversity loss and intra-style consistency loss. Subsequently, DoP employs visual domain learning (VDL) to further refine the target-domain-relevant style token generated by TDL by making use of the test visual data with self-supervised learning. Extensive experiments on popular DG benchmark datasets (e.g., PACS, VLCS, OfficeHome, and DomainNet) and ImageNet show that our DoP consistently outperforms the state-of-the-art DG approaches. Codes are available at: https://github.com/shiming-chen/DoP.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.