One Pool Is Not Enough: Multi-Cluster Memory for Practical Test-Time Adaptation

The stream clusterability analysis is methodologically sound and establishes a firm foundation for the proposed approach. The GMM-based diagnostic framework (measuring imbalance ratio, entropy, and mode coverage) directly links memory quality to downstream performance, showing MCM maintains near-optimal balance (imbalance ratio ≈1.8) while SCM fluctuates between 10–40×. The ablations are thorough: ACC outperforms global and LRU consolidation strategies; pixel-level descriptors outperform CNN features by over 10 percentage points; and the recurring TTA experiments demonstrate that MCM not only prevents collapse but actually improves over 20 rounds (32.8% at Round 20 vs. 33.3% at Round 1).

The reliance on pixel-level channel statistics (Eq. 2) assumes domain shifts manifest primarily in low-level appearance, which the authors acknowledge may fail for geometric transformations or high-level semantic changes. The heuristic for K_max = min(5, max(1, ⌊N_c/20⌋)) is dataset-dependent despite claims of being "tuning-free"—it effectively couples cluster capacity to label space size without theoretical justification beyond empirical observation. While ACC improves over alternatives, restricting consolidation to temporally adjacent pairs (reducing search from O(K²) to O(K)) is a strong inductive bias that assumes temporal correlation implies distributional similarity; the paper could stronger motivate why non-adjacent clusters should never merge.

The evidence supports the central claim that memory organization trumps capacity: Figure 3 shows that scaling SCM from 64 to 320 samples yields negligible improvement while increasing runtime 5×, whereas MCM with the same total capacity achieves substantially lower error. Comparisons to related work are fair—MCM is positioned as orthogonal to methods like TRIBE (which achieves better single-pass CIFAR-10-C results via tri-net architecture without memory) and is appropriately not compared directly on memory-free terms. The claim that gains stem from structure rather than capacity is validated by the diagnostic analysis: MCM maintains full mode coverage while SCM periodically loses entire modes.

The paper provides substantial implementation detail: per-cluster capacity N=64, distance threshold τ=0.3, and the K_max formula are specified. Experiments use single NVIDIA RTX 4090 GPU with RobustBench preprocessing. All three base methods (RoTTA, PeTTA, ResiTTA) are publicly available, and MCM is described as plug-and-play. However, no code or data repository URL is provided in the paper, and the exact localization of the distance threshold τ relative to descriptor scale could benefit from clearer specification—the appendix shows robustness across τ∈[0.1,0.7] but the physical meaning of these values in pixel-statistic space remains abstract. Runtime comparisons are reported but hardware-specific details beyond GPU model are omitted.