Machine Translation for Sign Languages
Sign language machine translation remains far behind spoken-language MT because sign languages exploit simultaneous manual and non-manual channels, spatial grammar, and three-dimensional construction. This survey reviews the technical arc from isolated recognition to end-to-end translation and production, embedding every stage in sign language linguistics and deaf-community ethics. The authors argue that progress must be judged not by benchmark scores alone but by whether deaf users actually accept and understand the output, making the manuscript as much a piece of technology assessment as a call for sociotechnical reform.
This is an unusually comprehensive and ethically grounded survey. The authors—an interdisciplinary hearing and deaf team—succeed in balancing computer vision advances with linguistic and stakeholder perspectives, producing a review that reads as a call for sociotechnical rigor rather than a simple literature inventory. Its central thesis, that optimizing benchmark metrics without community validation targets the wrong objective, is convincingly threaded through every section. The manuscript is descriptive rather than evaluative in places, but its framing around data sovereignty and community governance sets a higher standard for the field.
The linguistic primer in Section 2 is essential reading for vision researchers, clearly explaining simultaneity, classifier constructions, and spatial reference. The ethics section moves beyond generic AI principles to concrete governance requirements—accessible informed consent in native sign languages, fair remuneration, and NIL policies for avatar creation—grounded in deaf community position papers. Table 1 provides a valuable, compact overview of datasets across modality, language, and scale. The technical history is thorough, tracing the shift from gloss-based pipelines to gloss-free transformers and diffusion-based production without losing sight of why glosses were problematic in the first place.
As a review, the paper necessarily summarizes claims from primary sources without independent experimental verification, and it occasionally lapses into cataloguing architectures rather than critically synthesizing which design choices actually improve generalization. The authors note that gains are often tightly coupled to specific datasets, yet they do not perform a quantitative meta-analysis or systematically compare reported results under controlled conditions. Future directions in Section 6 are sensible—better evaluation, more conversational data, and modeling of non-manual features—but largely rehearse established gaps rather than proposing specific hypotheses or prioritization. The manuscript also lacks an explicit literature selection protocol, which makes it difficult to assess whether the coverage is systematic or shaped by author familiarity.
The evidence base is admirably broad, drawing on computer vision, sign language linguistics, and deaf studies. Comparisons with spoken-language MT are consistently explicit about modality differences—simultaneity, spatial grammar, and the absence of abundant text corpora—rather than treating sign languages as noisy speech. However, the technical evidence relies on reported benchmark numbers from individual papers without controlling for data leakage, compute, preprocessing pipelines, or annotation quality, an issue the authors acknowledge but do not correct. By contrast, the ethical argumentation is stronger because it synthesizes multiple independent community sources (WFD, BDA, COSET) and at least one global survey (Atwell et al.).
Because this is a review, reproducibility concerns shift from experimental replication to the transparency of synthesis. The authors do not state a systematic search or inclusion protocol, so it is unclear whether the surveyed literature is exhaustive or selectively curated. More importantly, the paper itself highlights why reproducing state-of-the-art sign language systems is so difficult: many core preprocessing steps—pose extraction, signer detection, background removal—introduce uncontrolled error, and benchmark datasets vary in annotation granularity from gloss-level to sentence-level only. The authors note that evaluation metrics like BLEU penalize valid paraphrases and that no agreed metric exists for production quality, meaning independent researchers cannot easily verify whether newer methods are genuinely better.
Sign language machine translation has progressed substantially over the past decade, evolving from isolated sign recognition to end-to-end translation systems. Advances in pose estimation, transformer architectures, and large-scale dataset collection have driven progress, yet challenges remain. Datasets are limited compared to spoken-language resources; evaluation metrics inadequately capture the linguistic quality of output; and models must capture the simultaneous, multi-layered, and three-dimensional structure of sign languages. This manuscript provides a comprehensive review that seeks to balance technical challenges with stakeholder considerations. We examine the linguistic properties that make sign languages computationally unique, trace the evolution of recognition, translation, and production systems, and analyze ongoing technical challenges. Crucially, we address ethical considerations around data governance, community involvement, and appropriate use. Drawing on interdisciplinary perspectives spanning computer vision, sign language linguistics, and deaf studies, our analysis emphasizes that continued progress requires sustained collaboration across these fields and with deaf communities.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.