A Systematic Comparison of Search Space Complexity and Efficiency for Time-Synchronous End-to-End ASR Models
Modern end-to-end ASR systems typically rely on simplistic fixed-size beam search, assuming that large search spaces are unnecessary. This paper systematically compares search complexity and efficiency for CTC and neural transducer models across three datasets, showing that dynamic score-based pruning adapts to local uncertainty and dramatically cuts the real-time factor without sacrificing accuracy. The work challenges the prevailing assumption that small fixed beams are sufficient and provides tuned decoding parameters for reproducibility.
A well-executed empirical study that convincingly demonstrates the value of dynamic pruning in time-synchronous E2E ASR decoding. The authors present strong evidence across diverse tasks that fixed small beams are suboptimal and that search spaces spike during uncertain acoustic regions. As stated in the introduction, "We challenge the assumption that small fixed-size beam search is sufficient for optimal performance with modern model architectures, in terms of both accuracy and speed." The paper is practical and actionable, though its scope is deliberately limited to offline time-synchronous models.
The analysis of search space behavior is rigorous and well-supported. The paper documents that "The p50 value shows that the median active hypothesis set is small while the p90 and p99 statistics show that in some rare difficult regions the signal causes locally increased hypothesis counts" (Section V), reinforcing the central argument with percentile data and clear visualizations. The efficiency gains are substantial—dynamic pruning reduces RTF by up to 94.6% in some configurations (Table IX)—and the authors admirably release code, algorithms, and tuned parameters to enable reproduction.
The title promises a "systematic comparison," yet the study excludes label-synchronous architectures such as AED and Speech-LLMs, which the authors admit: "This study is limited to time-synchronous CTC and transducer search; label-synchronous and streaming decoding require some fundamentally different analysis and remain future work" (Section VII). Additionally, the critique of other toolkits (Flashlight, ESPnet, k2, NeMo) is entirely qualitative—no shared benchmarks are run—so claims about relative limitations are not empirically validated here. Finally, the hardest acoustic evidence comes from an in-house Spanish telephony corpus that readers cannot access.
The internal evidence broadly supports the core claims. Tables III–VI show that low fixed beams incur high search error rates, while Figures 2–4 and Table IX demonstrate that dynamic pruning preserves WER with large RTF reductions. However, the comparison to related work is uneven. The authors discuss GPU-based transducer decoding optimizations (e.g., WIND, NGPU-LM, NeMo) but do not benchmark against them, noting only that those approaches "prohibit word-level processing and dynamic pruning." Without measured baselines from other frameworks, readers must trust feature lists rather than numbers. The authors also acknowledge that "A natural direction for future work is to extend this comparison to larger ASR models trained on more data, where integration of external LMs may have lower impact" (Section IV), which tempers the generality of the findings.
Reproducibility is mostly strong but not without friction. The authors state that "Our setups are published on GitHub" (Section I) and provide compact algorithmic descriptions plus Table X with tuned decoding parameters. All experiments use the open-source RASR toolkit, which is publicly available. Yet RASR is a niche framework compared to ESPnet or NeMo, posing a practical barrier to quick independent verification. Moreover, the Spanish telephony results rely on an undisclosed in-house test set, making that portion of the study unreproducible for external researchers. Details on hardware (A10 GPU) and the exact LM architectures are given, but training hyperparameters are summarized rather than exhaustively listed.
Search is a central component of Automatic Speech Recognition (ASR) systems and has been deeply investigated for classical ASR modeling based on Hidden Markov Models (HMMs). However, in work on current sequence-to-sequence ASR architectures it is often simplified to a small fixed-size beam search without external Language Model (LM). In this work, we present a comparative analysis of search space complexity and efficiency for Connectionist Temporal Classification (CTC) and neural transducer models under different pruning strategies and LM configurations. We show that these end-to-end (E2E) models work with a small search space most of the time but exhibit large temporary peaks in hypothesis count during periods of uncertainty. Score-based pruning dynamically adapts the search to model uncertainty, leading to a significantly lower Real-Time Factor (RTF). Our results highlight the limitations of search with low fixed-size beam pruning and provide practical guidelines for efficient search in E2E ASR.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.