AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark
AnyGameRenderer tackles the programmer-art problem of agent-built games by converting crude gameplay video into stylized, high-fidelity output via a rank-128 LoRA on the 33B MiniMax-H3 ref2va model. It accepts plain video inputs---gray blockouts, colored primitives, semantic maps, or wireframes---without engine G-buffers or per-object labels, and aims to preserve entity count, timing, and position across genres from tower defense to kart racing. The authors assemble roughly 10k paired clips from three sources and introduce an entity-level benchmark of 753 annotated entities to measure correspondence, improving temporal match over the zero-shot base.
The paper makes a timely, well-scoped contribution by framing generative game rendering as an entity-correspondence problem and backing it with a purpose-built benchmark. Yet absolute performance is modest: the hallucination rate remains 46.0%, real-world recordings barely improve, and the detection pipeline itself yields a 34.1% hallucination rate on ground-truth RGB video. The authors report that "Compared with the base model, AnyGameRenderer better preserves entity correspondence: temporal match rises from 50.9% to 62.9%, missed count changes fall from 34.7% to 21.3%, and fewer entities are hallucinated or left unrendered," but acknowledge that "Real recordings improve the least: temporal match rises from 46.5% to 51.1%, the unrendered rate is similar, and missed count changes stay at 20.0%."
The entity-level benchmark and four dedicated metrics are a genuine methodological advance over frame-level edge agreement. The caption schema, called the correspondence contract, is carefully designed: captions are programmatically generated so that, as the authors put it, "it teaches the model to decide what exists from the video, not from the text." The three-source data strategy is also strong, especially the Unreal Engine scenes explicitly scripted around failure modes: "Count changes in same-color groups. Groups of 2 to 20 characters walk in, spawn, walk out, or despawn during the clip, in dense clusters, columns, or scattered formations. This targets crowds left as unrendered blocks and entities that appear late or never."
Despite headline improvements, the model is still far from robust entity correspondence. The hallucination rate of 46.0% means nearly half of detected entity-frames are spurious, and the GT floor of 80.9% temporal match and 34.1% hallucination shows the SAM3-based evaluation pipeline is itself noisy. On real gameplay---the most practical setting---temporal match reaches only 51.1% and missed count changes do not improve at all. The paper also lacks direct quantitative comparisons to concurrent systems and provides no ablation studies on data mixture, caption design, or LoRA rank, leaving it unclear which design choices matter most.
The evidence supports that LoRA fine-tuning improves over the frozen base on the proposed benchmark, with synthetic Unreal data showing the clearest gains (temporal match 96.6% vs GT floor 98.5%). Claims about related work are factually accurate: CWM relies on a "proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video," while PWM uses "state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation." Unfortunately, no direct comparison with these concurrent systems is attempted, and the paper does not demonstrate how frame-level metrics used by prior work would score the same outputs, making cross-paper reconciliation impossible.
The authors share code and a project page, and training hyperparameters are disclosed: rank-128 LoRA, peak learning rate between $2\times10^{-5}$ and $2.8\times10^{-5}$, batch size 16, and 20-step Euler sampling. However, full reproduction is constrained: the 10,330 training clips draw on 38 proprietary GEAR games, custom Unreal Engine scripts, and a 3D-box detection pipeline whose exact parameters are not specified. The base 33B MiniMax-H3 model was only recently released under a community license. As reported, "Training runs on 16 NVIDIA H200 GPUs (141 GB). Each step takes about 129 s, and three epochs over all 10,330 training clips take about 48 hours," placing independent replication out of reach for most labs.
AI coding agents can now build small playable games in minutes, but the results usually look like programmer art: gray blockouts, colored geometric primitives, and flat semantic blobs. We present AnyGameRenderer, a video-to-video generative renderer that turns such footage into high-quality visuals with characters, motion detail, and effects. Throughout, it keeps the layout, camera, and every entity's position and timing intact. Our AnyGameRenderer generalizes across a wide range of genres that agents build, such as tower defense, auto-battlers, top-down survivors, turn-based battles, kart racing, and crowd runners. AnyGameRenderer takes plain video as its game-state input, with no complex representations such as engine G-buffers, per-object class labels, or low-poly proxies. Gray blockouts, colored primitives, semantic maps, and wireframes all work as input. It also preserves entity correspondence: group counts stay correct, and entities appear and disappear on the right frame. We implement AnyGameRenderer by LoRA fine-tuning MiniMax-H3 ref2va, a 33B reference-video-conditioned video generation model. Training uses $\sim$10k paired clips from three sources: (i) multi-stream rendering data from 38 agent-made games exported from GEAR, an AI game-creation tool; (ii) two-stream rendering data from Unreal Engine, with programmatically controlled crowds, count changes, and entity scales; and (iii) real gameplay recordings, with pairs reconstructed via 3D-box detection. To evaluate the entity correspondence, we propose an entity-level benchmark of 59 held-out cases with 753 entities, scored against ground-truth boxes. Compared with the base model, AnyGameRenderer better preserves entity correspondence: temporal match rises from 50.9\% to 62.9\%, missed count changes fall from 34.7\% to 21.3\%, and fewer entities are hallucinated or left unrendered.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.