Decision Titan: Test-Time Training for Long-Term Memory in Offline Reinforcement Learning

The strongest aspects of the paper are its careful ablation studies and mechanistic analysis. The authors visualize the learned forgetting gate $\alpha_t$, momentum $s_t$, and write gate $\eta_t$ over time, convincingly showing that successful runs rely on exactly one head maintaining high momentum decay $\gamma \approx 0.95$ to propagate gradient signals across chunks. They also honestly report that increasing the number of MLP heads improves success not by adding capacity but by increasing the probability that random initialization produces a usable pattern, which is a nuanced and useful observation. Additionally, the introduction of X-Maze as a controllable testbed for ordered sequential memory is a solid methodological contribution.

The paper suffers from a significant empirical gap: there is no direct comparison against contemporary RL long-term memory baselines such as RATE or ELMUR on shared tasks, making it impossible to assess whether DTi represents practical progress. The authors acknowledge that the architecture learns dependencies through chance patterns in the memory module, and the training curves show binary all-or-nothing behaviour where runs either rapidly converge or remain at random-chance levels; this brittleness raises serious questions about reliability in stochastic environments.

Moreover, the training protocol requires processing entire padded episodes, which the authors admit is unsustainable for longer environments and already constrains batch size, yet no remedy is offered. Catastrophic forgetting is identified as the most likely limiting factor, but it is neither quantified nor mitigated, leaving the approach vulnerable to corruption from every chunk-level gradient update.

Within the constrained scope of X-Maze, the evidence supports the specific claims about context-window multiples and embedding effects: the authors show that chunked embeddings generalize better than global ones because they avoid out-of-distribution positional indices, and the one-hot-repeat encoding yields perfect recall because queries reliably match keys. However, comparisons to related work are almost entirely theoretical. The complexity analysis correctly notes that DTi operates in $O(k^2 + p)$ versus ELMUR's $O(k^2 + km)$ and RATE's $O((k+m)^2)$, but without matching experiments these asymptotic arguments are speculative. The authors fairly note that ELMUR achieves 100% success on T-Maze corridors up to 1,000,000 steps while DTi fails at 200 steps, a gap they attribute to catastrophic forgetting; this honesty is commendable but also highlights that the empirical head-to-head comparison is missing and that the theoretical advantage in memory scaling does not yet translate to better extrapolation.

Appendix B documents hyperparameters clearly, and the training procedure—processing chunks in order without backpropagation through time—is described with sufficient detail to allow replication. The authors state they build on an unofficial Titans implementation by Wang (2025), which introduces an external dependency that may drift or become unsupported; they do not indicate whether their own code or the X-Maze environment will be publicly released. The experiments use a small model (32 dimensions, 3 layers) and a modest compute budget (1/8th of an Nvidia A100), making reproduction accessible, but exact replication of the chance learning dynamics and gate visualizations likely depends on the specific random seeds and the unofficial codebase. Reproducibility would be substantially improved by releasing code, environment source, and random seeds.