DISC: Domain-Integrated Similarity Cost for Time-Series Dataset
Time-series dataset similarity is task-dependent: no single metric reliably predicts downstream transfer error across models and protocols. This paper proposes DISC, a learnable aggregation framework that calibrates a base mixture of standardized similarity metrics with a rank-based prior gate. On a shared 46-dataset split, DISC raises the average source-wise Pearson correlation with forecasting error by about 10% over the best single metric, supporting the claim that learned integration is a stronger proxy for transfer behavior than manual metric selection.
The paper makes a clear case that dataset similarity for time-series transfer should be treated as a task-calibration problem rather than a fixed metric choice. The proposed DISC framework is well-motivated, cleanly formulated, and the empirical gain over single metrics is consistent across four task settings on the reported split. However, the evidence rests on a single train-validation-test partition with only 14 test sources and 13 target points per source-level correlation, so the magnitude and statistical reliability of the 10% average improvement remain uncertain. The authors themselves note that these results are based on one shared split, which tempers the strength of the claim but also reflects commendable honesty.
The conceptual framing is strong: Figure 1 and the accompanying analysis convincingly illustrate why single metrics disagree and why an integrated approach is needed. The base-plus-prior gating design is interpretable, and Table V shows that the learned weights sensibly vary across tasks---e.g., OOD-Time-MoE relies on MAD and DBA-LCSS, while OOD-Lag-Llama uses PCA-DTW and PCA-LCSS. The manual-search ablation in Table VI further supports that learning is necessary, as an exhaustive non-learned search over roughly 7.9 million configurations falls well below DISC on every task.
The experimental evidence is thin on statistical robustness. Each source-wise correlation in the test split is computed from only 13 target datasets, making Pearson correlation highly variable and sensitive to outliers. The study uses a single shared split (24 train / 8 val / 14 test), so there is no estimate of variance across data partitions, and the authors explicitly caution that the results should be interpreted as alignment under the shared split rather than a repeated-split uncertainty analysis. Additionally, the downstream evaluation conflates target intrinsic difficulty with source-target compatibility---larger DISC scores may simply track harder targets rather than mismatched sources---yet the paper frames this as intentional rather than attempting to disentangle them.
The paper evaluates DISC against a comprehensive set of ten single-metric baselines (dataset-level and reducer-based) and a strong non-learned manual-search baseline, which is fair and thorough within the paper's scope. The gain over the per-task oracle best single metric ($0.543 \to 0.598$) shows that aggregation captures complementary signal beyond simply selecting the strongest individual metric. However, the paper does not compare against recent transferability-estimation methods such as those cited for time-series foundation models (e.g., Zhang et al., 2026; Yao et al., 2025), so it is unclear whether DISC improves over task-specific transferability predictors that use model-internal signals rather than dataset similarity metrics.
Reproducibility is hampered by the absence of any mention of code release, data preprocessing scripts, or exact foundation-model checkpoint versions in the provided text. The paper lists hyperparameters in Table I, but critical details such as similarity metric implementations (e.g., whether WSD uses the Gaussian approximation from Chen et al., 2025), the specific 46 GluonTS datasets used, and the random seeds for restarts are not specified. Because the method requires training-time access to directed downstream errors on calibration datasets, independent reproduction would need the exact same source-target error matrix; without open-source release, this barrier is substantial.
The growth of time-series datasets and foundation models has made dataset comparison a central part of time-series workflows. Assessing source-target transfer before model adaptation requires similarity estimates that can summarize transfer-related dataset relationships without exhaustive source-target evaluation over every candidate pair. Many metrics have been proposed for measuring similarity between time-series datasets, but they capture different dataset relationships and vary in how well they align with downstream utility across tasks. We evaluate this gap by measuring the correlation between source-target similarity scores and downstream forecasting errors in out-of-distribution and transfer-learning settings. We propose Domain-Integrated Similarity Cost (DISC), a learnable aggregation framework that uses calibration transfer observations to select, weight, and combine similarity metrics for different downstream tasks. By integrating complementary metric values, DISC better distinguishes source-target dataset differences that matter for downstream behavior. After calibration, DISC scores source-target pairs using only their similarity-metric values and yields higher alignment than single-metric references on the reported split. These results show that time-series dataset similarity is task-dependent, and that learned metric integration provides a stronger proxy for downstream transfer behavior than manually selecting one similarity metric.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.