Have an LLM Write Your Anomaly Detector: Autonomous Discovery of Compact, Interpretable Detectors for Time Series
Instead of hand-designing or training a neural network for time-series anomaly detection, this paper outsources the job to an LLM-driven autonomous research loop. The agent iteratively edits a short NumPy program, scoring candidates against synthetic anomalies in a held-out training slice, until it discovers compact spectral-Gaussian novelty detectors. The resulting univariate and multivariate programs reportedly top the TSB-AD benchmark while running on a single CPU, offering an unusual blend of accuracy, speed, and interpretability.
The paper presents a compelling proof-of-concept for LLM-guided algorithm discovery, and the resulting detectors are both elegant and empirically strong on the primary benchmark. However, its boldest claims rest on a single stochastic search run per regime, a synthetic validation signal that may favor the discovered structure, and comparisons against untuned baselines that do not receive the same meta-optimization budget. The idea is promising, but the evidence is not yet robust enough to conclude that LLM-authored detectors are generally superior.
The text itself tempers the headline: 'the detector's standing depends on the metric family and the benchmark.'
The two discovered detectors are methodologically clean. The univariate detector stacks eight spectral scalars into $\phi(x) \in \mathbb{R}^{19}$ and scores them with a shrunk Mahalanobis distance $s_w(x) = \sqrt{z(x)^\top \hat{\Sigma}^{-1} z(x)}$, while the multivariate detector pools per-channel band-powers into a single descriptor and uses a squared Mahalanobis distance $d(\phi) = (\phi - m)^\top \Sigma^{-1} (\phi - m)$. The ablation in Table 3 isolates the descriptor's contribution: substituting the canonical catch22 feature set while keeping the same novelty framework drops VUS-PR from 53.9 to 28.4. The authors also report weaknesses honestly, such as the mid-field affiliation F-measure on TAB.
The strongest concern is that each reported detector is the product of a single, unrepeatable LLM session. The paper states that 'Each detector emerged from a single one hour session of roughly 100 edit-and-evaluate iterations,' providing no variance estimates, repeated runs, or sensitivity analysis. Because LLM sampling is stochastic, the reported performance may reflect a fortunate trajectory rather than a reliably discoverable algorithm.
A second concern is fairness of comparison. The search optimizes against synthetic anomalies drawn from the GutenTAG taxonomy, which supplies an implicit supervisory signal tailored to the benchmark. None of the classical or deep baselines are tuned with this signal, so the comparison is not on equal footing. The paper also notes that the ShinkaEvolve evolutionary baseline, despite an identical prompt, 'instead converged in both regimes on a hacky weighted ensemble of standard methods,' suggesting the search procedure can subvert the intended single-principle objective.
The quantitative evidence on TSB-AD is extensive, but not uniformly persuasive. The headline claim of 'most first-place finishes on every reported metric' is tempered by the TAB results, where 'On univariate TAB it leads the ranking-oriented ROC metrics, while KMeans is strongest on the precision metrics,' and where the affiliation F-measure is mid-field. Baselines are run from released checkpoints without tuning, whereas the discovered detector benefits from an LLM-driven meta-optimization on synthetic anomalies that match the benchmark taxonomy. This creates an asymmetry: the baselines are not given the same validation signal or search budget.
Where the evidence is strongest is in the rediscovery check (Table 3). The paper shows that holding the Mahalanobis-novelty framework fixed and replacing the descriptor with catch22 features collapses performance, supporting the claim that the specific spectral-organization signature is the source of gain rather than the general framework.
The final NumPy programs are fully disclosed in Appendices C and D and in an anonymized repository, which satisfies artifact transparency for inference. Reproducing the discovery process is another matter. The exact LLM prompts, temperature settings, and context window management are not documented, and the proprietary models (Opus 4.8, GPT 5.6 Terra) may evolve. The paper emphasizes that 'Each detector emerged from a single one hour session of roughly 100 edit-and-evaluate iterations,' but a single stochastic run means independent reproduction could easily land on a different program with different performance. The Karpathy-style agentic loop concept is grounded in existing open-source work, yet without a frozen container or API snapshot, the search trajectory is effectively irreproducible.
Time-series anomaly detection trades off predictive accuracy, computational efficiency, and interpretability. We use a large language model not as the detector but as the author of one: an autonomous research loop in which the model repeatedly edits a single short NumPy program under a leakage-free objective, keeping the best-scoring detector it finds. The loop discovers two compact detectors, one for univariate and one for multivariate series, that describe short windows by their local spectral features and compare them with the training-region distribution through a covariance-aware distance. On the TSB-AD benchmark these detectors lead the field across metrics, ahead of the strongest classical, deep, and foundation-model baselines including Time-RCD, yet they train no network and use no GPU, and the multivariate detector is faster than every similarly performing baseline. LLM-driven program search is thus a practical route to accurate, efficient, and transparent detectors.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.