Energy and Latency First: How Deployment Constraints Reverse Time-Series Model Selection
This paper studies handover prediction in cellular networks, where time-series models must meet a hard 40 ms inference deadline and a tight energy budget on user equipment. Rather than ranking by accuracy alone, the authors propose a protocol that filters models by tail latency feasibility then ranks survivors on measured per-inference energy—including feature computation. They benchmark eight architectures and find that gradient-boosted trees dominate the accuracy–energy–latency frontier, while the top accuracy-only neural model (CNN-1D, F1=0.914) costs over 3× more energy and trails in F1. The work challenges standard benchmarks by showing that duty-cycled measurement and full-pipeline accounting can invert model rankings.
The paper makes a timely and well-supported argument that accuracy-only benchmarks can invert deployment suitability for edge time-series tasks. The evaluation protocol is clearly motivated by the cellular use case, and the empirical finding that LightGBM outperforms neural models on both accuracy and energy is demonstrated across controlled ablations. However, the claimed generalizability beyond telecommunications rests on three informal conditions that are not independently validated, and the measurement is performed on server-class rather than handset silicon. As the authors note, "Accuracy-only benchmarking does not merely under-report deployment cost. It can rank first a model that cannot be deployed."
The experimental design is a clear strength: measuring full-pipeline latency and energy under both busy-loop and realistic duty-cycled schedules directly addresses a gap in prior work. The decomposition of per-inference cost into preprocess, forward, and postprocess is methodologically sound, and the ablations—feature versus architecture, capacity sweeps, and distillation—robustly isolate sources of advantage. The authors show honesty in reporting that duty-cycled measurement narrows LightGBM’s energy advantage from 10.4× to 3.3×, writing that "Busy-loop benchmarking therefore does not merely omit the duty-cycled idle cost; it inflates the apparent energy advantage of the cheapest model."
Several limitations temper the headline conclusions. First, energy is estimated from CPU utilization at 50 ms resolution rather than hardware power counters, so absolute microjoule figures are approximate and the idle-floor estimates may be noisy. Second, the dataset is a single 100-second ray-traced urban scene with a known closed-form labeling function, which inherently favors the handcrafted A3-aware features fed to LightGBM; although the raw-window ablation shows trees still win, the real-world generalization to noisier field deployments remains untested. Third, the three stated conditions for generalizability—a simple decision boundary, short sequences, and duty-cycled inference—are plausible but never validated outside the telecommunications setting, making the broader claim speculative. The authors acknowledge the measurement limitation explicitly: "Energy is estimated from CPU utilization rather than hardware counters, so μJ are estimates while relative rankings on one platform hold."
The evidence strongly supports the core claim within the studied setting: every neural architecture is dominated by LightGBM on the measured frontier, and neither distillation nor a capacity sweep closes the accuracy gap. The comparison to related work is fair in noting that prior handover literature evaluates accuracy alone and that FLOPs correlate poorly with device energy. However, the negative result on distillation—the student F1 fell from 0.438 standalone to 0.407 distilled—may be specific to the small teacher-student configuration tested, and the paper does not rule out that more sophisticated compression could change the ranking. The authors are careful to note that "the gap is a capacity floor, not a tuning artefact," though this is asserted rather than proven for all possible neural approaches.
Reproduction is partially supported but blocked by the absence of an explicit code release or persistent data repository. On the positive side, hyperparameters, random seeds ({42,123,456,789,1024}), model architectures, and the feature engineering recipe are documented in detail across the appendices, and the dataset generation pipeline using Sionna and SUMO is described sufficiently for an independent implementation. The energy measurement protocol is clearly specified, but because it relies on a specific 36-core Xeon server and CPU-utilization-based power sampling, independent labs would need identical hardware to replicate the absolute microjoule numbers; relative rankings are more transferable. Without access to the exact generated dataset or codebase, reproducing the full benchmark would require substantial reimplementation effort.
AI models, and time-series models in particular are typically ranked by accuracy and deployed under constraints absent from that ranking. In this paper, we study a real setting where these constraints are explicit and measurable: handover pre- diction in cellular networks, where inference runs on the User Equipment once per 40 ms measurement period and must complete before the network acts. The timing bound is hard: a late prediction is superseded by the measurement it was predicting, while energy is a cumulative budget spent for the life of the device. Neither appears in the objective these models are selected on. We evaluate eight architectures under a protocol that screens for deadline feasibility at the tail rather than the mean, then ranks the feasible models on measured inference energy, count- ing the feature computation that precedes the forward pass and is usually omitted. Gradient-boosted trees on handcrafted temporal features are the most accurate model (F1=0.974) and among the cheapest to run, while the neural architecture an accuracy-only benchmark would rank first (CNN-1D, F1=0.914) costs over 3 times more energy per inference under a realistic duty-cycled deployment pattern. We give three conditions under which this outcome should be expected, so the finding applies beyond telecommunications. Accuracy-only benchmarking does not merely under-report deployment cost. It can rank first a model that cannot be deployed.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.