Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance
Mobile manipulation demands precise coordination between a mobile base and a robotic arm, making real-world policy fine-tuning difficult because of high-dimensional, tightly coupled action spaces. This paper proposes HiRe-MoMa, a framework that freezes a pre-trained base policy and learns a small, hierarchical residual: one head corrects the base motion while the other corrects the arm conditioned on the base correction. It also guides learning with sparse human teleoperation interventions via an auxiliary critic-ranking loss, reporting large success-rate gains on a TIAGo++ robot after roughly 30 minutes of online interaction.
HiRe-MoMa is a practical and well-engineered real-world fine-tuning system for mobile manipulation. The hierarchical residual architecture and policy-agnostic design are sensible, and the reported sample efficiency is impressive. However, the experimental validation is narrow—single platform, single operator, no seed sweeps—and the core technical novelty over existing residual RL and human-in-the-loop RL is modest. The paper claims it is "among the first frameworks to enable efficient on-robot fine-tuning for mobile manipulation in the real world," but comparable contemporaneous methods already demonstrate similar real-world fine-tuning of large policies without residual decomposition.
The decomposition ablation provides strong evidence that splitting the residual into independent base and arm heads is the dominant driver of performance, confirming that gradient interference in monolithic action spaces is a real bottleneck for mobile manipulation. The policy-agnostic residual paradigm is elegant because it sidesteps intractable log-likelihoods of generative base policies and works across architectures. The real-world sample efficiency is striking: the full system raises average success from 39.2% to 75% within 25–40 minutes of online interaction.
First, the unidirectional conditioning head adds only a modest average gain (+5 percentage points) over decomposition alone, and its effect is task-dependent—negligible on the single-arm Place Only task—so its impact is arguably overstated relative to the broader hierarchical-framing claims. Second, the expert-guided value alignment loss is essentially a Q-value ranking objective; the paper does not isolate whether its benefit comes from the ranking formulation or simply from digesting extra expert data, because ablations against behavior cloning on the same intervention buffer are absent. Third, the experimental scope is very limited: one robot, one teleoperator, 20 trials per condition, no seed sweeps, and the hardest task (Pick Place) is omitted from the build-up ablation due to cost. The authors also note that remaining failures are caused by hardware issues such as sticky casters and a narrow arm workspace, which undermines claims of broad real-world robustness.
The comparison against +Flat Residual (equivalent to HIL-SERL) is intrinsically favorable because HIL-SERL was designed for fixed-base arms and lacks any cross-module structure; a mobile-manipulation-specific baseline is missing. The +HG-DAgger baseline is effectively a straw man, since it is well known to overfit to sparse corrections. The paper also does not quantitatively compare against contemporaneous policy-agnostic RL methods such as PA-RL, which fine-tunes diffusion policies and a 7-billion-parameter VLA model autonomously in the real world in 40 minutes. While the ACT result shows that HiRe-MoMa works across architectures, it tests only two imitation-learning backbones and does not address very large VLA models that represent the current frontier.
The authors provide detailed appendices covering network architecture, SAC hyperparameters, and the warm-up buffer protocol, which is commendable. However, no public code or pre-trained base policies are available at the time of review, and reproduction would require access to a TIAGo++ robot, a Meta Quest 3 teleoperation stack, and a trained human operator. The evaluation uses only 20 trials per condition with no repeated random seeds, so statistical robustness is unclear. The residual bound ϵ is tuned manually, and the success of the method depends on a small warm-up buffer of successful on-robot rollouts that may not be easy to obtain for new tasks.
Mobile Manipulation (MoMa) is a cornerstone of autonomous robotic systems, yet real-world fine-tuning remains a formidable challenge because of high-dimensional action spaces and intricate coupling between heterogeneous modules. In this paper, we propose HiRe-MoMa (Hierarchical Residual Mobile Manipulation), a systematic framework for stable and sample-efficient real-world fine-tuning of complex MoMa systems. At its core, HiRe-MoMa adopts a Hierarchical Residual Architecture that fine-tunes a frozen base policy. A Policy-Agnostic Residual learns additive corrections atop the base policy, while Hierarchical Decoupling splits the residual into mobile-base and manipulator-arm heads with unidirectional conditioning, mitigating gradient interference and aligning corrections with the platform's physical hierarchy. To improve sample efficiency in data-expensive real-world settings, we propose Expert-Guided Value Alignment, which leverages sparse human interventions with under 20% training coverage to reshape the critic's value landscape, effectively guiding autonomous exploration through expert priors. Extensive real-world experiments on the TIAGo++ platform demonstrate that HiRe-MoMa consistently enhances task performance with only around 30 minutes of online interaction, regardless of the base policy architecture. To our knowledge, this is among the first frameworks to enable efficient on-robot fine-tuning for mobile manipulation in the real world, providing a practical paradigm for deploying complex robotic systems in dynamic, human-centric environments. Further details can be found on our project website: https://hire-moma.github.io/
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.