Bilingual Language Models Capture Cross-Linguistic Influence in Second-Language Reading
Monolingual language models predict native reading times well but miss how a reader's first language shapes second-language processing. This paper investigates whether bilingual GPT-2 models sequentially trained on an L1 and then English can capture human cross-linguistic influence during L2 reading. The authors evaluate 55 models---monolingual and bilingual with varying steps of L1 exposure---against eye-tracking data from 12 L1 backgrounds, showing that L1-matched bilingual surprisal improves prediction of late-stage reading measures and peaks for intermediate-proficiency learners. The work is significant because it provides a computational bridge between multilingual NLP and psycholinguistic theories of non-native sentence processing.
The paper makes a compelling, carefully controlled case that bilingual language models encode L1-specific biases predictive of human L2 reading behavior. Its strongest contribution is the matched-versus-mismatched bilingual comparison, which isolates language-specific transfer from generic multilingual gains. However, the study is restricted to English as the L2 and relies on a single model initialization, leaving questions about typological generality and training stability unanswered. As the authors note, "we trained each model configuration using a single initialization seed. Although this precludes measuring variance across different weight initializations, we mitigate this limitation through the breadth of our experimental setup."
The experimental design is a clear advance over related work. By including both monolingual baselines and L1-mismatched bilingual controls, the authors credibly attribute predictive gains to L1-specific transfer rather than mere multilingual capacity. The results are theoretically coherent: the L1 advantage is absent in early measures like first fixation duration and strongest in late measures such as go-past time and total fixation duration. The script-overlap analysis further strengthens the account, with the authors observing that "when languages share a script, visual input can directly activate orthographic representations in both languages simultaneously, allowing cross-linguistic effects to emerge in early reading measures."
Despite large-scale training, the study suffers from limited statistical power at the model level because every configuration uses a single random seed, meaning observed differences could partly reflect initialization luck rather than systematic L1 transfer. Several interpretive claims are also stretched thin. The script analysis groups languages by Latin versus non-Latin orthography, but as the authors acknowledge, "this is only one axis of variation," and other typological factors such as word-order alignment and morphological similarity are uncontrolled confounds. Additionally, the inverted-U proficiency effect depends on LexTALE scores the authors themselves call into question: "its validity as a measure of global proficiency has been questioned, as it is fundamentally a lexical measure," and the PCA robustness check shows attenuated significance for total fixation duration.
The evidence largely supports the core claim. The nested model comparisons show that "matched models contribute substantially more unique log-likelihood and achieve statistical significance at much higher rates than their mismatched counterparts," and the native English control consistently falls below the L2 mean, confirming specificity to non-native processing. Comparisons to prior work are fair and precise; the authors correctly identify that earlier studies lacked monolingual baselines, froze L1 representations, or evaluated only early metrics. Still, the high pairwise surprisal correlations between bilingual models (above 0.94) mean the additional variance explained is small in absolute terms, making the practical significance of the L1-matched advantage modest despite its statistical reliability.
Reproduction is theoretically possible but practically hindered by missing artifacts. The architecture, hyperparameters, and data sources (FineWeb, MECO L2) are documented in detail, including a single shared seed of 123 across all 55 models. However, no code or model checkpoints are mentioned as released, and the sentence-level filtering pipeline used to minimize code-switching---while described---is not accompanied by scripts or thresholds. Because each model requires training up to 64k steps on 1B tokens, the compute barrier is substantial, and without released weights or intermediate checkpoints, independent verification of the exact surprisal values would demand full retraining from the same single seed.
While monolingual language model (LM) surprisal is widely established as a predictor of reading times, it cannot capture the non-native (L2) reading dynamics that are shaped by the reader's first language (L1). In this work, we investigate whether the cross-lingual transfer within bilingual models aligns with human L1-driven biases during L2 reading. We evaluate bilingual models on eye-tracking data of readers from 12 diverse L1 backgrounds reading English texts. To isolate L1-specific transfer, we compare bilingual models trained on an L1 that matches the readers' L1 background to both a monolingual English baseline and bilingual models trained on other L1s. We demonstrate that matching the readers' L1 improves the prediction of L2 reading times. This predictive advantage is concentrated in late-stage eye-tracking measures, modulated by L1-L2 script overlap, and peaks among intermediate-proficiency readers. These findings provide evidence that bilingual LMs capture aspects of cross-linguistic influence, and that such influence is present in L2 reading behavior.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.