Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation
Existing retrieval-augmented MT applies retrieval uniformly, yet LLM parametric knowledge suffices for many inputs and external evidence is only conditionally useful. This paper proposes SRAT, an agentic RL framework that treats reasoning, retrieval, and translation as learnable actions optimized via GRPO for final translation quality. It learns when to retrieve, what to query, and how to integrate evidence, supported by two retriever-alignment strategies and evaluated across idiom, knowledge-intensive, and document-level settings. The work is important because it challenges the default assumption that retrieval is always beneficial and shows that adaptive, quality-aligned evidence acquisition consistently outperforms fixed pipelines.
SRAT is a well-motivated and empirically strong paper. The partial-evidence framing correctly identifies that uniform retrieval is suboptimal for translation, and the agentic RL formulation—interleaving reasoning, retrieval, and translation as a single policy—yields consistent gains over never- and always-retrieve baselines across three benchmarks and multiple language directions. Utility-aligned retriever training and black-box translator experiments further strengthen the contribution, though limitations in human-evaluation scale and remaining headroom versus an oracle policy temper the conclusion.
The paper's experimental design is a clear strength. The introduction of SRAT-N and SRAT-A as matched ablations isolates the value of the adaptive retrieval decision from other training effects, and SRAT consistently outperforms both on semantic metrics. The black-box translator experiments are compelling: training a small Qwen policy to control retrieval for frozen Claude and GLM models improves translation quality, showing the retrieval policy can be decoupled from the generator. The contrastive retriever alignment is particularly effective when semantic similarity is insufficient, raising retrieval accuracy on RAGtrans En→De from 72.6% to 81.4% and improving MQM by 7.4%.
Several limitations deserve attention. First, SRAT underperforms supervised fine-tuning baselines on BLEU because the RL reward optimizes COMET and LLM judges, not n-gram overlap; while the authors defend this, it means gains are metric-dependent. Second, an oracle per-instance selector between never- and always-retrieve policies reveals remaining headroom, especially on RAGtrans where the oracle retrieves less often than SRAT (30.4% vs 43.0% on En→De) yet achieves better MQM, suggesting the learned policy is not yet fully calibrated on retrieval necessity. Third, the human evaluation is small (32 examples), and the authors acknowledge that retriever alignment is translator-specific, limiting cross-model transfer. Finally, increasing retrieved documents beyond top-1 consistently degrades quality, indicating that current utility alignment does not robustly exploit larger evidence sets.
The evidence broadly supports the claims. Main results are reported across multiple automatic metrics (BLEU, COMET, MQM, and task-specific LLM judges) and three diverse datasets, reducing the risk of dataset-specific overfitting. The authors conduct robustness checks with alternative LLM evaluators (Opus 4.8 and DeepSeek 3.2), a stronger Qwen3-8B backbone, and multiple online search backends, all preserving the relative system ranking. Comparisons to related work are fair: prior systems are accurately characterized as using fixed routing or optimizing retrieval separately from translation, and the authors explain why translation differs from QA-oriented retrieval (evidence supports rather than contains the answer).
Reproducibility is partially strong but has gaps. The authors report detailed hyperparameters (GRPO with KL coefficient 0.001, clip ratio 0.2, group size 8, sampling temperature 1.0), hardware (A100 80GB GPUs), training duration in GPU hours, and random seeds (three seeds). Full LLM-as-judge prompts are provided in Appendix F, and all datasets are publicly available. However, the paper does not mention a public code release or artifact repository, which would be necessary for independent reproduction of the RL training loop, the SGLang tool-calling integration, and the exact VeRL configuration. Without released code, independent reproduction of the agentic trajectory sampling and retrieval-shaping curriculum would be difficult.
Retrieving external knowledge can improve large language model (LLM) translation by providing supplementary information such as idiom interpretations and terminology. However, existing retrieval-augmented machine translation systems typically apply predefined retrieval strategies uniformly across all inputs. We argue that translation operates in a partial-evidence regime: LLM's parametric knowledge suffices for many inputs, while external evidence benefits a subset and usually supports rather than directly provides the desired translation. This creates two challenges: deciding when to retrieve and what evidence to acquire. We introduce Selective Retrieval-Augmented Translation (SRAT), an agentic reinforcement learning framework that learns when to translate directly or retrieve, what to retrieve, and how to integrate these decisions by optimizing final translation quality. To align evidence retrieval with downstream translation quality rather than simple semantic similarity, we further develop joint Plackett--Luce optimization and translation-quality contrastive training for retriever-training. Experiments on idiom, knowledge-intensive, and document-level translation with offline or online retrieval show that SRAT consistently outperforms never- and always-retrieve baselines, and quality-aligned retriever training provides further gains when similarity-based retrieval is insufficient. The results demonstrate the benefit of treating retrieval as a learnable translation action rather than fixed preprocessing.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.