Group Relative Policy Optimisation Improves Multilingual Speech Recognition in Low-Resource Languages

The core methodology is sound. The authors isolate the reinforcement signal through a well-designed ablation showing that "[t]he same data is thus useful under a reinforcement objective and harmful under a supervised one" (Section 4.1). The headroom-based clip selection is a thoughtful practical innovation: they keep clips where $g=\mathrm{median}_i\ \mathrm{CER}(\hat{y}_i,y)\;-\;\min_i\ \mathrm{CER}(\hat{y}_i,y)$ is largest, i.e., where the model has capacity but not reliability. The characterization of improvement mechanisms is also insightful: "GRPO repairs these decoding failures" where the SFT model falls into loops, and on non-looping utterances it makes "near-miss spelling" errors rather than acoustic hallucinations (Section 4.2).

The headline results deserve scrutiny. "Best published macro-averaged WER" is technically correct, but macro-averaging across languages with different morphologies can hide poor per-language performance. On AfriVox-v2 (Table 1), SunflowerASR loses to Sahara v2 on Swahili, Kinyarwanda, Yoruba, and Akan, and to Gemini 3 Flash on several others. The practical deployment evaluation relies on only 60 clips across two languages, making reported percentage improvements hard to trust statistically. The paper also does not explain why 5 of 40 languages regress with GRPO, or why CER was chosen as the reward metric while WER is the reported benchmark metric. Computational cost of generating $G=10$ rollouts per example is not discussed, which matters for reproducibility and adoption.

A broader conceptual gap is whether freezing the Whisper encoder is optimal. The authors justify this on the hypothesis that GRPO "is most effective at correcting decoding errors," but they do not ablate an unfrozen encoder, leaving it unclear how much of the available headroom remains unexplored.

The comparisons to related work are mostly fair. The key claim that prior GRPO work was limited to LLM-decoder architectures is accurate: Shivakumar et al. used "Auto-regressive, causal, decoder only LLMs" with acoustic encoders feeding into an LLM. However, the benchmark comparisons against Gemini, Sahara v2, and OmniASR LLM-7B are apples-to-oranges comparisons across vastly different model sizes, training data, and paradigms, without controlling for these confounders. The paper does not establish that Whisper-large-v3 (1.55B) is meaningfully "smaller" than the 7B speech LLMs it contrasts with. On SimbaBench (Table 2), OmniASR LLM-7B achieves lower WER than SunflowerASR on multiple languages (Swahili 0.156 vs. 0.246; Amharic 0.234 vs. 0.600), suggesting that model size and pretraining still dominate for some languages.

The trained model is publicly released on HuggingFace and the training corpora are described, which aids reproducibility. However, critical details are missing: the total number of GRPO optimizer steps or epochs is not stated (only that the extra SFT ablation ran for "one further epoch"), random seeds are not provided, and no code repository is linked. The large batch requirement—64 distinct clips with $G=10$ rollouts for a batch size of 640 at the decoder—is identified as important, but this creates a practical barrier for researchers without access to substantial GPU memory. The per-language training data ranges from 5.2 hours to 1,392 hours, yet the headroom selection抽出 only 15,285 clips total for GRPO, a tiny fraction of the 3.82M training clips; the paper does not justify why so few clips suffice or whether scaling this set would improve results.