Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
67 papers in cs.RO
Trending mixes fresh papers with community signal.
0
cs.AIcs.RO Diyuan Shi, Yiqi Tang, Hongyin Zhang et al. · Oct 14, 2026

Vision-language-action models enable instruction-conditioned robot manipulation, but they typically cannot improve from experience after deployment. This paper proposes VLRLA, a framework that equips frozen VLA models with in-context reinforcement learning by conditioning action generation on a recurrent memory of past observations, actions, rewards, and termination signals. Using algorithm distillation on constructed learning histories, VLRLA aims to gradually improve task success during deployment without updating network parameters.

Vision-language-action (VLA) models connect visual perception and language understanding with robot control, providing a foundation for instruction-conditioned manipulation. However, conventional pretrained VLA policies lack a mechanism for gradual adaptation through accumulated interaction experience during deployment. We therefore propose VLRLA, a general framework that equips existing VLA models with \textit{in-context reinforcement learning} capabilities for deployment-time adaptation. Our approach augments a VLA model with a reinforcement learning module that learns from historical observations, actions, rewards, and termination signals. During deployment, the policy gradually refines its action generation and progressively improves task success while keeping model parameters fixed. Our approach is also designed for general applicability and compatibility with diverse learning history construction strategies and model training pipelines. Finally, through extensive experiments, we systematically investigate the learning properties of VLRLA in simulation and demonstrate its ability to adapt to unseen scenarios during real-world deployment.
0
cs.ROcs.AI Yiquan Li, Aidan Bradshaw, Chenfeng Gao et al. · Oct 14, 2026

Wrist-worn IMUs power everything from gesture recognition to clinical motor assessment, yet most prior benchmarks evaluate each task in isolation, leaving unclear whether a single pretrained representation can serve all of them. This paper introduces IMU-Bench, a unified evaluation of ten tasks spanning kinematic estimation, semantic retrieval, activity recognition and Parkinson's assessment, and proposes InrtlWrist, a masked autoencoder pretrained on recorded and simulated wrist motion. The core finding is that pretraining yields broadly useful features—leading on nine of ten frozen-probe tasks—but that supervision from one downstream task rarely helps unrelated ones, distinguishing universal representation quality from task compatibility.

Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimation, inertial odometry and clinical motor assessment. Existing wearable benchmarks evaluate performance across multiple tasks, but leave open how supervision for one task affects transfer to others and whether these tasks benefit from a shared representation. We present IMU-Bench, a benchmark of ten tasks from eight datasets for studying IMU multi-task performance and cross-task transfer. We compare six self-supervised methods pretrained on unlabeled recorded and simulated wrist motion, evaluating classification, pose and velocity estimation, and IMU-language retrieval. We find that our masked autoencoder, InrtlWrist, achieves the best or joint-best frozen-probe performance on nine of ten tasks, and fine-tuning improves on training from scratch on nine of ten tasks, including full-body pose, odometry and retrieval. Across transfer and joint-training experiments, supervision that benefits one target can reduce performance on others, with supervised sources transferring within kinematic estimation while, under frozen probing, none approaches the pretrained encoder on language retrieval, and joint-training gains that come at a cost to the paired task. These findings distinguish broad downstream usefulness from complementary task supervision. Code, task cards and a leaderboard are publicly available.
0
cs.AIcs.CLcs.RO Minghao Guo, Qingyue Jiao, Xiujin Liu et al. · Oct 14, 2026

Multimodal agents need persistent memory to act coherently over long horizons, yet forming and managing memory across heterogeneous, high-dimensional observations—video, audio, 3D geometry, and GUIs—has lacked a unifying analytical lens. This survey introduces a two-axis lifecycle framework that classifies how experience is represented (source, textual, structured, latent, or parametric) and how memory states and policies evolve ($T1$ Static, $T2$ Dynamic, or $T3$ Self-Evolving). By analyzing 297 architectures, the authors identify key trade-offs between fidelity and efficiency and expose the striking scarcity of systems that evolve their own memory policies at deployment time.

Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a significant challenge as they move toward long-horizon tasks and lifelong settings. Addressing this challenge requires effective memory mechanisms that can maintain continuity across sessions, adapt to changing users and environments, and refine memory-management policies through accumulated experience. We introduce a two-axis analytical framework for multimodal agent memory structured around two key questions: how multimodal information is formed and represented (Axis I, Formation and Representation), and how its management adapts with accumulated experience (Axis II, Memory Management). Axis I, Formation and Representation, characterizes what information is retained and how it is encoded (e.g., modality-specific embeddings, textual and structured records, latent and parametric representations) to support subsequent reasoning and action. Axis II, Memory Management, distinguishes three tiers: T1 Static Memory reuses fixed memory; T2 Dynamic Memory updates memory through a stable policy; and T3 Self-Evolving Memory updates memory through an evolving policy. Based on this framework, we synthesize recurring design patterns and derive principles for balancing source fidelity with abstraction, maintaining temporal validity and provenance, grounding memory use in recoverable evidence, and evolving memory policies over time. Furthermore, we discuss the challenges and potential future directions for the memory mechanism of multimodal agents.
0
cs.ROcs.CV Yunze Liu, Jing Wang, Shenrui Wu et al. · Oct 14, 2026

Robota-Embed tackles the problem of retrieving relevant physical experience for robots by learning a single embedding space that jointly encodes vision, language, and action from both the current state and a future trajectory segment. The core insight is that a useful retrieval key must capture not just task intent and visual scene, but also the robot's ongoing motion and execution stage, because the goal is to retrieve an appropriate continuation rather than simply a similar observation. A new multi-source benchmark and two downstream applications—in-context policy conditioning and memory evolution around a frozen large policy—show that this unified, action-aware representation significantly outperforms vision-language-only baselines and improves both success rates and token efficiency.

Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware retrieval space can support robot in-context learning and task-local context evolution.
0
cs.ROcs.AIcs.CV Yue Zhang, Han Lin, Zun Wang et al. · Oct 14, 2026

Wedge addresses the inefficiency of generic environment generation for embodied navigation: pretrained agents fail for different environmental reasons, so simply adding more diverse or realistic training scenes may not address their actual shortcomings. The paper proposes a policy-conditioned generation framework that diagnoses an agent's specific weaknesses by contrasting its successful and failed rollouts, uses an LLM to propose executable environmental features (such as target visibility or blind-path ratio), validates them via success-rate gaps, and generates targeted training environments. This approach is tested across three heterogeneous agents—an RL policy, an imitation-learned model, and a black-box VLM—showing consistent gains in ProcTHOR and transfer to HM3D.

Simulated environments enable scalable training data generation for embodied agents, but more data does not necessarily provide what a particular agent needs to learn. Existing environment-generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons. We therefore introduce WEDGE, a framework for weakness-guided navigation learning in which the failure modes of a pretrained agent guide the generation of training environments at each adaptation round. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success-rate gap. The validated weaknesses are then mapped to the environmental factors that control them. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness-conditioned adaptation consistently outperforms matched random and hard-example alternatives on ProcTHOR, while the resulting gains for Uni-NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re-diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.
0
cs.CVcs.CLcs.RO Kyusong Lee, Heting Ying, Tiancheng Zhao · Oct 14, 2026

VLX-Go targets embodied visual tracking in distractor-rich shared spaces by casting local motion as vision-language-conditioned short-horizon waypoint prediction. It combines frozen DINOv3 and SigLIP features, multiscale temporal context, and a transformer backbone to predict $K=8$ robot-centric waypoints. A staged training recipe—vision-language alignment, multimodal reasoning mid-training, and tracking specialization—drives a large improvement over the OmTrackVLA baseline, while preliminary DAgger fine-tuning provides modest further gains. This matters because existing methods suffer severe success-rate drops when moving from single-target to distractor tracking.

Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractors remains challenging. Existing methods exhibit a substantial drop in success from single-target to distractor tracking, motivating a focus on the shared spaces that person-following robots must navigate. We present \method{}, a vision--language waypoint policy built around the hypothesis that visual and language representations should be aligned before tracking-specific motion is learned. The policy combines complementary frozen visual encoders, multiscale temporal context, a transformer backbone, and a lightweight head for robot-centric waypoint prediction. Its training progresses through vision--language alignment, multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes. Compared with 41.54\% distractor-tracking success for our earlier OmTrackVLA model, the complete multi-stage recipe achieves 55.80\%. We then explore on-policy refinement using filtered oracle labels for states visited by the policy. This preliminary study adds 2.42 percentage points of distractor-tracking success, reaching 58.22\%, while reducing collision rate by 1.99 pts. The final policy records the highest distractor- and ambiguous-tracking success rates among the compared methods. These results support staged multimodal learning as a promising direction for tracking among distractors and motivate further study of on-policy refinement for person following in shared spaces.
0
cs.CVcs.RO Hochul Hwang, Jahir Sadik Monon, Soowan Yang et al. · Oct 14, 2026

NavAble tackles a critical data scarcity problem in assistive navigation: standard vision datasets largely ignore accessibility-critical objects such as pedestrian signals and door-activation buttons, leaving blind and low-vision assistive systems with unreliable perception. The authors introduce a large-scale dataset and a synthetic generation pipeline that combine 36K curated real-world images, 5.5K newly collected egocentric frames, and 452K synthetic frames across 11 BLV-relevant classes. By filling this gap, NavAble provides a foundation for training segmentation models that can recognize navigationally important objects from diverse assistive-device viewpoints.

Reliable recognition of accessibility-critical objects (e.g., audible pedestrian signals, door-activation buttons, and handrails) is essential for assistive navigation technologies that support safe, independent mobility for blind and low-vision (BLV) users. Yet these categories are underrepresented in existing large-scale vision datasets, and the few datasets that include them suffer from limited class coverage, inconsistent annotations, and poor diversity. We introduce NavAble, a large-scale dataset spanning 11 accessibility-critical object classes, combining 41K curated open-source images with 8K newly collected, densely annotated real-world images. To extend scale and diversity, we develop a synthetic data generation pipeline that renders high-fidelity 3D assets across 37 environments under diverse styles and viewpoints, yielding 452K frames with rich ground-truth annotations from 565 assets. The pipeline minimizes manual effort by combining automated web image crawling, VLM-based filtering, SAM 3D-based asset generation, and convenient camera trajectory configuration, enabling large-scale data generation at a diversity unattainable through manual collection. Experiments show that fine-tuning existing segmentation models on the NavAble Dataset achieves reliable accessibility-object segmentation (>80% mIoU on our held-out test set). We also demonstrate the effectiveness of synthetic data augmentation, which yields up to +3.5 mIoU over real-world-only fine-tuning. Beyond benchmark evaluation, validation on egocentric data from two mobility-assistive robots demonstrates improved perception reliability in real-world navigation scenarios. Together, NavAble establishes a scalable foundation for BLV navigation perception, directly addressing the data scarcity and embodiment gaps. The dataset and pipeline are publicly available at https://huggingface.co/datasets/NavAble/NeurIPS_2026_BLV.
0
cs.ROcs.LG Michael Y. Fatemi, Jinhao Liang, Ferdinando Fioretto · Oct 1, 2026

Multi-agent motion planning with diffusion models currently demands costly training on large feasible-trajectory datasets, making learned planners map-specific and data-hungry. This paper introduces TFDP, a training-free diffusion-style planner that replaces global neural score networks with analytical local scores derived from smoothness, obstacle-avoidance, inter-agent separation, and velocity-limit terms. By exploiting the spatial and temporal locality of trajectory refinement, the method scales to hundreds of agents and obstacles in seconds without any demonstrations or neural network training.

Path finding and multi-robot motion planning require trajectories that are smooth, goal-directed, and collision-free in environments with complex geometric constraints. Recent diffusion-based planners have shown that trajectory generation can be cast as iterative denoising which has opened the doors to learning-based approaches that can handle multi-modal trajectory distributions and refine entire trajectories. However, a key limitation is that diffusion planners require training on large collections of feasible trajectories, rendering them map-specific, and difficult to deploy when high-quality demonstrations are unavailable. This paper introduces a training-free diffusion-based motion planner that replaces learned global trajectory scores with analytical local scores derived from obstacle, smoothness, velocity, and inter-agent feasibility terms. The proposed idea relies on a key observation: the score of a trajectory can be reconstructed by considering only local interactions between neighboring waypoints and nearby constraints. This structure exploitation yields a decomposed denoising procedure that retains the optimization structure of classical trajectory methods while inheriting the iterative refinement behavior of diffusion models. Experiments on a large collection of complex environments and large multi-agent planning tasks show that the proposed analytical score produces smooth and feasible trajectories within limited computational costs, for example in generating feasible paths for 300+ agents in environments containing 100+ obstacles in under 6 seconds on a GPU, outperforming strong learning-based and optimization baselines, while avoiding the data requirements of learned diffusion planners.
0
cs.ROcs.AIcs.HC Amr Mousa, Rifny Rachman, Neil Karavis et al. · Oct 1, 2026

Quadrupedal locomotion requires balancing command tracking, stability, and energy efficiency, yet conventional RL bakes these trade-offs into a static scalar reward that cannot change after training. PROMO proposes a preference-conditioned multi-objective framework that takes the relative priority vector $w_t \in \Delta^{K-1}$ with $K=3$ as a runtime input to a single policy, while keeping embodiment-specific locomotion priors fixed. On a Unitree Go2, the same policy transferred zero-shot modulates behavior through preference changes alone, cutting specific energy by up to $30.4\%$, position error by $38.7\%$, and peak body-attitude deviation by $59.0\%$.

Quadrupedal locomotion requires balancing conflicting objectives such as command tracking, stability, and energy efficiency, yet conventional reinforcement learning (RL) hardcodes these priorities into a fixed scalar reward at training time. We present PROMO (Preference-Conditioned Multi-Objective Reinforcement Learning), a semantic multi-objective approach that makes this trade-off an explicit runtime input to a single locomotion policy. PROMO conditions the policy on deployment facing preferences while keeping embodiment-specific locomotion priors fixed, thereby separating operator intent from reward shaping terms required for viable gait generation. Compared with fixed-objective controllers, multi-objective baselines, and independently trained specialists, PROMO achieves objective specialization and robustness from a single deployable policy. Across 100 sampled preferences in simulation, 67 behaviors are non-dominated under exact Pareto dominance, with a mean preference-objective correlation of 0.843, demonstrating broad Pareto coverage and predictable preference response. The same policy transfers zero-shot to a Unitree Go2, where preference changes alone reduce specific energy by up to 30.4%, position error by 38.7%, and peak body-attitude deviation by 59.0% relative to the balanced preference. These results establish preference-conditioned multi-objective RL as a practical runtime interface for adaptive legged locomotion, extending its role beyond offline Pareto-set construction. Open-source code and videos are available at https://amrmousa.com/promo/.
0
cs.LGcs.AIcs.RO Sebastian Sanokowski, Alireza Sarmadi, Majid Khadiv · Oct 1, 2026

Policy-gradient methods suffer from high variance, while pathwise actor updates require accurate action derivatives that standard value-function training does not guarantee. FERPO addresses this by deriving an entropy- and KL-regularized target action distribution from critic values alone, then fitting the policy with forward KL via self-normalized importance sampling---never differentiating the critic with respect to actions. The method targets massively parallel on-policy robot learning, where reliable policy improvement is critical for both sample and wall-clock efficiency.

Several state-of-the-art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic. However, critics are typically trained to predict returns, and accurate value predictions do not necessarily yield accurate action derivatives, potentially leading to unreliable policy updates. We propose Forward Entropy-Regularized Policy Optimization (FERPO), an on-policy maximum entropy reinforcement learning algorithm that performs policy improvement using critic values without differentiating the critic with respect to actions. FERPO derives an optimal target action distribution from a policy-improvement objective regularized by entropy and Kullback-Leibler (KL) divergence. We then fit the actor to this target by minimizing a forward-KL objective, estimated using self-normalized importance sampling (SNIS) with actions drawn from the rollout policy. By limiting the target distribution's deviation from the rollout policy, the KL regularization helps keep these importance weights well behaved. In contrast to reverse-KL objectives, which can favor a subset of the target distribution's modes, the forward-KL objective encourages coverage of multiple high-value modes and thereby promotes exploration. Experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample-efficiency gains. Computational benchmarks also demonstrate faster actor updates than Relative Entropy Pathwise Policy Optimization (REPPO).
0
cs.ROcs.CVcs.LG Kartik B. Kapse · Oct 1, 2026

Autonomous driving system designers face a persistent dilemma: should they build a decomposed modular pipeline or train an end-to-end neural policy? This survey-style paper addresses that question by comparing modular, imitation-learning, reinforcement-learning, and hybrid architectures across interpretability, scalability, and robustness. Its central contribution is a practical twelve-criterion, four-dimensional framework intended to guide practitioners in selecting the right paradigm for a given deployment context.

Autonomous driving systems have become a central focus of intelligent transportation research, with End-to-End Learning and Modular Architectures offering two prominent design paradigms for their implementation. E2E Learning uses deep learning algorithms to map raw sensory inputs directly to driving actuators, providing a streamlined and adaptable solution. while Modular Architectures employ a pipeline-based approach, dividing the system into distinct subsystems for perception, cognition, planning, and control. This paper presents a comprehensive comparative analysis of these paradigms, focusing on their strengths, limitations, and trade-offs to provide insights into their suitability for various autonomous driving applications. The study evaluates key factors such as interpretability, scalability, robustness, and real-world applicability. While End-to-End Learning emphasizes simplicity and adaptability in dynamic environments, it lacks transparency and is highly dependent on large datasets. Conversely, Modular Architectures offer superior interpretability and task-specific optimization, but face challenges related to integration complexity and scalability. To address these limitations, hybrid approaches that combine the strengths of both paradigms have emerged, offering a promising direction for overcoming these challenges. Beyond this comparative synthesis, following work proposes a Four-Dimensional Architecture Selection Framework, comprising twelve binary criteria across safety, operating environment, data/computational resources, and deployment context, and validate it against ten published autonomous driving systems, correctly recommending 7/10 deployed architectures. This work synthesizes existing literature to highlight key trade-offs between the paradigms and identifies hybrid architectures as a promising direction for future research.
0
cs.ROcs.AIcs.LG Seungyeon Kim, Junhoo Lee, Baekseung Kim et al. · Oct 1, 2026

World Action Models (WAMs) jointly predict visual futures and robot actions, yet they often hallucinate locally plausible trajectories that never execute the critical state transition needed to finish the task. This paper identifies "task-incomplete imagination" as a failure mode arising from the mismatch between long-horizon video pretraining and short-horizon control generation, and proposes Completion Aware Guidance (CAG), a training-free sampling method that up-weights instruction tokens already attended to by the model. Across two WAM architectures, CAG boosts success rates and cuts incomplete-imagination failures, offering a lightweight inference-time fix for a structural objective mismatch.

World Action Models (WAMs) predict visual futures and robot actions, yet they remain susceptible to task-incomplete imagination, where plausible, action-consistent predictions omit the transition needed for task completion. In this paper, we show that this failure is not inherent to the world model backbone, but emerges when adapted for short-chunk control, which can repeatedly favor plausible local continuations over task-completing transitions. To address this, we introduce Completion Aware Guidance (CAG), a training-free sampling method that guides generation toward task completion. Across representative WAMs, CAG improves success from 64% to 70% on a RoboTwin 2.0 subset and from 69% to 75% in zero-shot simulation, while reducing task-incomplete imagination from 79% to 40%.
0
cs.LGcs.RO Takumi Hara, Kanata Suzuki · Oct 1, 2026

Latent world models plan by scoring action sequences with latent-space distances, yet physical success criteria demand precision finer than these models typically encode. This work proposes a training-only auxiliary loss that supervises encoder and predictor outputs against the exact success-criterion quantities via a shared linear head, then discards the head at test time so planning remains unchanged. On manipulation tasks the approach yields statistically significant absolute gains of 3-5%, with the key insight that a task's success criterion itself should dictate what the latent state must retain.

Latent world models plan by scoring candidate action sequences with distances in latent space. However, task success is judged by physical quantities, which we call the success-criterion quantities. In all four latent world models we examine, the end-effector position is encoded in the latent state with an error larger than the success criterion allows. Such a latent state cannot separate successful candidates from failing ones. We propose an auxiliary loss that uses success-criterion quantities as training targets, whereas existing latent world models take them only as inputs. During training, a linear head on the encoder and predictor outputs regresses the success-criterion quantities, and the regression error is added to the training loss. The head is discarded after training, so the model, its cost, and its inputs at test time are unchanged. This loss alone improves the success rate on PushT and cube by 3.5% and 3.4% (absolute), respectively, and both improvements are statistically significant. A success criterion thus specifies what a world model must retain in its latent state, and we show that it can serve directly as a training target.
0
cs.MAcs.AIcs.LG Andrea Iannoli, Ivan Zyrianoff, Angelo Trotta et al. · Oct 1, 2026

Unmanned aerial vehicle swarms operating with independent edge-deployed language models face a dual challenge: reasoning context degrades as interaction histories grow, and indiscriminate information sharing wastes limited bandwidth. This paper proposes a distributed architecture in which each UAV maintains structured atomic notes in bounded core, local, and peer-specific memory queues, coupled with a deterministic Interest-Aware Gossip Engine that forwards messages based on semantic novelty and recency. The work matters because it targets the practical gap between centralized LLM coordination, which introduces bottlenecks and single points of failure, and fully decentralized control, which risks context degradation and communication collapse.

Unmanned aerial vehicle (UAV) swarms increasingly rely on language-model agents to provide adaptive mission-level reasoning in uncertain environments. Fully distributed control, in which each UAV hosts an independent Small Language Model (SLM), removes reliance on a centralized coordinator but introduces an information-management problem: long-running interaction histories can degrade the reasoning context, while indiscriminate information dissemination increases communication and inference overhead. We address these challenges with a distributed UAV-agent architecture that enables continuous local SLM control through an event-driven reason-act-observe lifecycle. Runtime knowledge is represented as structured atomic notes and organized into core, local, and peer-specific memory. A deterministic interest-aware gossip engine selectively disseminates these notes according to recipient-specific semantic novelty and recency. We evaluate the architecture using ten UAVs in a simulated search-and-rescue mission. Our approach completes all experimental runs, whereas unrestricted flooding messages completes only 70-85\%, and delegating forwarding decisions to the SLM prevents mission completion in every run. Compared with unrestricted flooding, our approach approximately halves inference-token consumption, reduces transmitted data, and achieves lower survivor-count error.
0
cs.ROcs.LG Giulio Schiavi, Andrei Cramariuc, Michael Pantic et al. · Oct 1, 2026

Standard grasp synthesis systems freeze their parameters after offline training and struggle with unfamiliar objects. This paper presents a continual-learning framework that replaces the fixed parametric scorer with a memory-based non-parametric model in a learned embedding space, augmented by recalling and transferring user demonstrations to new scenes via geometric registration. Instead of gradient-based finetuning, the robot adapts by adding grasp outcomes and demonstrations to structured memories, which enables fast, lightweight updates during real-world deployment with limited forgetting.

Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90\% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.
0
cs.ROcs.LGcs.MA Yifan Hu, Luhang Hong, Mingkang Long et al. · Oct 1, 2026

Coordinated multi-humanoid loco-manipulation is notoriously difficult because it couples high-dimensional whole-body control with decentralized decision making and scalability. MASkillBlender addresses this by learning a shared decentralized high-level multi-agent RL policy that blends frozen, pre-trained single-humanoid primitive skills, eliminating the need for task-specific motion references or painstaking per-task reward engineering. The authors further introduce a permutation-based data augmentation strategy and prove, under a homogeneous Markov game formulation, that it preserves the policy-gradient direction. Simulated experiments across cooperative manipulation and locomotion tasks suggest broad versatility across robot embodiments and simulators, though the work stops short of real-world validation and is limited to small, homogeneous teams.

Coordinated multi-humanoid loco-manipulation is promising yet challenging due to high-dimensional whole-body control, decentralized decision making, and scalability. While recent reinforcement learning methods have improved single-humanoid whole-body control, extending them to the multi-humanoid setting remains nontrivial and often requires substantial reward engineering or task-specific design. We propose MASkillBlender, a general multi-agent reinforcement learning framework to achieve decentralized multi-humanoid whole-body coordination. By learning a shared decentralized high-level policy over reusable pre-trained single-humanoid skills, MASkillBlender enables coordinated behaviors using only task-level rewards, without requiring task-specific motion references. To improve learning efficiency, we further introduce a permutation-based data augmentation strategy for homogeneous multi-humanoid systems, and theoretically show that the permuted samples preserve the policy-gradient direction of the original samples under the homogeneous Markov game formulation. We evaluate MASkillBlender on multiple multi-humanoid coordination tasks across two humanoid embodiments. Simulation results demonstrate that the proposed framework consistently achieves strong task performance and enables coordinated behaviors across different tasks and humanoid embodiments.
0
cs.LGcs.RO Oliver Obst, Frieder Stolzenburg · Oct 1, 2026

In model-predictive control with the cross-entropy method, a learned world model scores candidate action sequences and refits the Gaussian proposal for the next search iteration, creating planner-induced query shift. The authors disentangle these two roles by saving every CEM candidate pool from four surrogate models, executing all candidates in a resettable environment for a ground-truth ranking, and rescoring each pool with every model in a crossed design; they then splice the first elite set with environment-ranked elites to trace causal consequences through later iterations. Their results show that standard proposal distances contract even as ranking agreement stays near chance and mean disagreement grows, and that a single environment-guided elite update lowers final selected cost—demonstrating that world-model quality must be judged by its effect on the proposal, not just on candidate scoring.

The cross-entropy method (CEM) uses world-model scores to select action sequences and fit the distribution sampled in its next iteration. A scoring error can therefore change both the present decision and the candidates considered later. We evaluate these two roles separately. Four types of predictive model generate CEM traces, and every model rescores every saved candidate pool. Executing the same candidates in the environment provides a reference elite set and proposal update. Across twelve independently trained task-seed units on Walker and Cheetah, the pre-specified proposal distance falls from the first to the final CEM iteration in every unit. Proposal widths contract and fitted means separate relative to the remaining search width. Pairwise ranking agreement stays near chance on Walker and declines on Cheetah; elite-set agreement does not improve. This comparison shows greater variation between scorers than between pool sources on Cheetah; Walker has variation in both and in their pairings. We use the original six units to select Random nonlinear for a one-update intervention, without inspecting intervention outcomes. Replacing its first model-ranked update with an environment-ranked update lowers final realised selected-sequence cost in those six units and in six further units held out from the selection.
0
cs.ROcs.AImath.OC Yuji Takubo, Daniele Gammelli, Marco Pavone et al. · Oct 1, 2026

Spacecraft rendezvous and proximity operations currently require specialized engineers to translate high-level mission intent into dynamically feasible trajectories, creating a scalability bottleneck for distributed space systems. This paper introduces OrbitTAMP, a hierarchical task-and-motion planning framework that leverages pretrained LLMs to map natural-language commands into a partial mission specification, then systematically completes unspecified behavior, timing, and waypoint decisions via a domain-specific behavior graph and planning modules. Final trajectory optimization ensures dynamical feasibility and safety, aiming to provide an auditable, scalable language interface for spacecraft operations.

Spacecraft rendezvous and proximity operations (RPO) are currently planned through an expertise-intensive process in which engineers translate high-level operational intent into safe, dynamically feasible trajectories, creating a bottleneck to scalable operations. Large language model (LLM)-based agents could offer an intuitive interface for this process, although their outputs are not inherently grounded in orbital dynamics, operational constraints, or the structure of admissible spacecraft maneuvers. To exploit their semantic reasoning while ensuring the generated plan's physical validity, this paper presents a hierarchical framework for spacecraft task-and-motion planning (TAMP) that grounds LLM reasoning in a graph of reusable behaviors and domain-specific planning modules. Within this framework, a pretrained LLM maps a natural-language command to a partial mission specification. The associated planners then resolve unspecified decisions within the admissible operational space. Finally, trajectory optimization converts the completed mission specification into a dynamically feasible trajectory. Numerical experiments demonstrate that this architecture substantially improves intent recovery over direct LLM generation, achieving 98% exact recovery of partial mission specifications across all evaluated splits when backed by frontier LLMs. Additional test-time-compute experiments show that, for a compact 9B model, verifier-guided revision increases exact recovery from 75% to 88%, while broader behavior-plan search independently improves selection among admissible trajectory realizations. Overall, these results establish a scalable and auditable foundation for language-driven agentic planning of spacecraft RPO.
0
cs.ROcs.AI Xuehui Yu, Eason Yu, Meiyi Wang et al. · Oct 1, 2026

Vision-language-action policies often fail on partially observable, long-horizon manipulation because the current image alone does not determine the correct action. This paper reformulates robotic memory as an optimisation problem: the optimal memory should maximise the conditional mutual information $I(a_t; m_t \mid o_t)$ between action and memory given the current observation. To realise this, the authors propose Divide-and-Remember (D&R), a recursive lightweight selector that retains the top-$K$ most action-relevant tokens from an unbounded history while staying in the pretrained VLA token space. On the RoboMME benchmark and in real-robot experiments, D&R achieves strong gains with budgets as small as 64 tokens, offering both theoretical clarity and empirical progress on memory-augmented visuomotor control.

Vision-language-action (VLA) models struggle on history-dependent manipulation tasks, where the current observation alone does not determine the action, and the policy needs a memory of the history. Existing memory methods decide what to remember by design, for example, keeping frames with large pixel changes, and show inconsistent gains across tasks. We view what to remember as an optimisation problem. From the POMDP formulation of imitation learning, we show that the optimal memory maximises the conditional mutual information $I(a_t; m_t \mid o_t)$ between the action and the memory given the current observation. Intuitively, this means preserving the action-relevant information in the history that is not already contained in the current observation. Based on our analysis, we propose Divide-and-Remember (D&R), a recursive memory method that learns a memory function $m_t = M(h_t)$ and scales to long contexts while staying compute-light. It involves two strategies: (1) the selection over the full history is divided recursively into subproblems of top-$K$ selection over $2K$ tokens, so that fixed-size, lightweight selectors learned end-to-end support an unbounded history; (2) all recursion blocks share one selector, which captures the selection rule common to every block and keeps the method efficient. On RoboMME, a benchmark of 16 long-horizon manipulation tasks that require remembering when, where, what, and how to act, D&R achieves a state-of-the-art average success rate with consistent gains across all four suites under a budget of only 64 tokens; real-robot experiments show the same gain. Code, checkpoints and more results are at https://dnr-memory.github.io/
0
cs.ROcs.AI Aly Magassouba · Oct 1, 2026

Screw Attention replaces abstract graph edges in policy transformers with rigid-body spatial algebra. Each token carries a body pose, and messages between tokens are transported by relative poses and joint screws, making the layer equivariant to independent frame changes at every link. This yields compact learned policies that outperform larger unstructured networks and remain robust to the kinematic description conventions that break existing architectures.

Learned manipulation policies rediscover from data the spatial relations that rigid-body mechanics supplies in closed form. This costs data, and it leaves the policies fragile to geometric changes in the scene. We present Screw Attention, a transformer layer in which the relation between two bodies is a spatial transform rather than a graph edge. Every token is a body with a pose. Each pair of tokens carries the relative pose and, for robot joints, the joint screw. Messages are transported along this relation into the receiver's frame, while the attention scores see only frame-invariant quantities. By construction, the messages are equivariant to an independent change of frame at every token, and a single layer can express the velocity recursion of rigid-body mechanics. On simulated manipulation tasks, Screw Attention matches or outperforms controls of the same size, including graph, transformer and flat networks on LIBERO-Spatial. With 16,162 parameters it reaches 97.3% on LIBERO-Spatial from object poses (without images or language), above a flat network with 27x more parameters. Under a change of per-link frame convention its success is unchanged, while every other learned network falls below 3%. Placed on an analytic controller as a gated residual, it raises insertion success by 17.3 points. It is unaffected by pose noise up to 10,mm and by joint offsets within the factory calibration of a Franka arm. These results suggest a criterion: geometry is decisive when the task requires relations between frames that no other part of the system supplies. Code and trained policies will be released.