Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
Evaluating robotic foundation models still relies on expensive closed-loop rollouts in simulation or reality. This paper proposes Sensorimotor Alignment ($S^2$), a training-free offline metric inspired by the Platonic Representation Hypothesis, which scores how well a model's sensory embeddings align with the geometry of recorded end-effector trajectories. The authors find positive correlations between $S^2$ and task success across vision-language-action models, raw vision-language models, world action models, and real-robot policies, and argue the metric can cheaply rank models and diagnose representation-level behavior.
The paper is conceptually elegant and timely: it extends representational convergence ideas to embodied AI and offers a concrete, computationally cheap metric. However, the empirical argument is far weaker than the framing implies. Every key correlation rests on tiny sample sizes (5--12 models), relies on one-tailed $p$-values that inflate apparent significance, and includes at least one undisguised case of post-hoc horizon selection for the real-robot result. While the authors candidly admit that the relationship is correlational, the main text still presents $S^2$ as an explanatory instrument and a proxy suitable for model selection, claims that the current statistics do not robustly support.
The conceptual framing is genuinely useful: grounding the Platonic Representation Hypothesis in physical motion gives robotics a new representational target, and the metric is cleanly defined as a centered kernel nearest-neighbor alignment between sensory and motor spaces. The robustness sweep is unusually thorough---the authors vary horizon, reference dataset, neighbor count, pooling, trajectory normalization, and DTW choices---and the sensitivity check under embodiment mismatch ($r$ dropping from $+0.999$ to $+0.246$ when the reference arm configuration mismatches) is a sensible sanity check that suggests the metric is not merely reflecting inductive biases.
The statistical foundation is alarmingly thin. The flagship VLA correlation rests on only 12 models ($r=+0.717$), the raw VLM correlation on 8 ($r=+0.681$, one-tailed $p=0.031$), the WAM correlation on 6 ($r=+0.718$, one-tailed $p=0.054$), and the real-robot correlation on 5 policies ($r=+0.831$, one-tailed $p=0.041$ at a post-hoc horizon). One-tailed tests are hard to justify when the sign is not pre-registered, no multiple-testing correction is applied across the many tested horizons and modalities, and the real-robot analysis explicitly abandoned the prespecified horizon after it failed to reach significance. The 'representation-level explanations' in Section 4 are equally fragile: the frozen-encoder claim is based on exactly two checkpoints, the modality-dominance story on 12 models where text-feature $S^2$ simply lacks discriminative variance, and the WAM denoising and mid-depth analyses on only 3 models. Framing these post-hoc observations as mechanistic explanations conflates correlation with causation.
The evidence supports a coarse trend, but the comparisons to alternative metrics are lopsided and the effect sizes are fragile. Aggregate VQA scores are generic visual-linguistic benchmarks with no action content, so their near-zero correlation with robot success is unsurprising and not a persuasive argument for $S^2$'s unique value. Likewise, video-quality metrics such as FID, SSIM, and PSNR measure pixel fidelity rather than motion geometry; beating them on 3 world action models is weak evidence. The paper never controls for obvious confounders such as model scale or pretraining compute, both of which likely drive both $S^2$ and downstream success. The in-sample confound controls (Appendix B.2) only leave out one architecture group at a time from the same 12-model cohort and cannot rule out these fundamental covariates.
The authors release code and provide detailed appendices on hyperparameters, dataset filtering, and metric variants, which is commendable. Independent reproduction is nevertheless nontrivial because the metric requires exact checkpoint extraction pipelines for a dozen distinct VLAs, eight VLMs, and six WAMs, many evaluated with proprietary or inconsistently reported protocols. The reliance on previously published success rates aggregated from disparate sources introduces uncontrolled variance in task sets, camera views, and random seeds. Reproducing the real-robot correlation would require the exact DROID subsample and anchor-sampling code; while available, any deviation in token pooling or trajectory horizon could shift the reported scores.
Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.