Player-1: Learning to Play from Synthetic Experience
Player-1 attacks a core bottleneck in game agents: large-scale demonstrations cover only a narrow slice of dynamic game states, while interactive training in commercial games is hard to scale. The authors extend the computer-use action space with relative pointer motion, sustained keypresses, and composite keyboard–mouse actions. They fine-tune a VLM on just 6,044 curated gameplay trajectories (about 16 hours of control), then run multi-environment PPO in sixteen lightweight synthetic 2D/3D environments with programmatic, verifiable rewards. The central claim is that this synthetic experience yields large Minecraft gains (+27.8 percentage points for the 9B model) and generalizes to unseen games such as ViZDoom and browser-based GameWorld.
The paper makes a solid conceptual contribution—using cheap synthetic environments with verifiable rewards to push game control beyond small demonstration sets—but the empirical case is weaker than advertised. The 9B model improves dramatically on MineDojo-300, yet the much larger 35B-A3B model barely beats it after RL (41.04% vs. 38.33%), despite starting from a far stronger SFT baseline. This undercuts the scalability narrative. Baseline comparisons are performed under the authors' own evaluation protocol, which limits comparability to published numbers. Moreover, RL actually degrades 9B performance on the ViZDoom mobile-combat category, indicating unstable transfer.
The design of Player-1-Gym is pragmatic and well motivated. By identifying recurring control demands—aiming, combat, driving, navigation, and UI interaction—and instantiating them in sixteen randomized synthetic environments, the authors sidestep the engineering nightmare of scripting rewards in Minecraft or commercial FPS engines. Per-environment advantage normalization for PPO is a sensible solution to reward-scale heterogeneity. The codebase and model weights are released, and hyperparameters and environment details are specified in the appendices.
First, the SFT dataset is tiny: 6,044 trajectories spanning roughly 16.4 hours of gameplay, annotated by a VLM rather than humans. This raises questions about reasoning quality and coverage. Second, RL training is extremely short—only 200 steps with 256 episodes per step—yet the authors draw broad conclusions about scaling and transfer. Third, the 35B-A3B model gains only 17.7 pp from RL versus 27.8 pp for the 9B model, and its final MineDojo success is only marginally higher, suggesting that the synthetic RL signal saturates quickly and fails to leverage additional capacity. Finally, the 9B model's ViZDoom mobile-combat score drops from 0.5293 after SFT to 0.3033 after RL, revealing clear negative transfer that the paper does not adequately explain.
The MineDojo-300 headline is striking, but the authors explicitly note it is measured 'under our evaluation protocol,' which means different screenshot resolutions, action intervals, and history settings than those used by competing papers. Because baseline models are re-evaluated in this bespoke setup, the numbers are not directly comparable to published leaderboards. The inclusion of GameWAM, JARVIS-VLA, and UI-TARS as baselines is fair, yet Table 1 lists Game-TARS and SIMA 2 as unavailable closed systems even though the paper never benchmarks Player-1 against them numerically, making the comparison somewhat moot. On GameWorld the gains are modest (+2.7 pp success for 9B), weakening the generalization claim for browser games.
Reproduction is tractable in principle because the project page, GitHub repository, and model weights are public. The appendices provide substantial detail on action semantics, mouse calibration, per-environment reward functions, PPO hyperparameters, and the environment-scaling protocol. However, reproducing the exact SFT data would require re-annotating 295,097 turns with Gemini 3.8 Flash (the authors do not release the raw annotated reasoning strings), and some calibration coefficients for demonstration datasets are recording-specific. The synthetic environments depend on internal 2D/3D engines and shared asset libraries that are not yet documented outside the code release, so independent replication hinges on the completeness of the open-sourced repository.
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.