Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
1579 papers
Trending mixes fresh papers with community signal.
0
cs.CVstat.ML Mingrun Jiang, Yuejia Liu, Zishan Shao et al. · Oct 1, 2026

Diffusion models with classifier-free guidance (CFG) compute paired conditional and unconditional activations at each denoising step, yet existing post-training quantizers treat those branches as independent sources. This paper identifies the CFG branch space as a structured coding axis—matched activations are strongly correlated and jointly shaped by the guidance direction—and proposes Guidance-Correlation Branch Transform (GCBT), a lightweight $2\times 2$ orthogonal rotation applied per layer before activation quantization. It admits a closed-form per-layer solution, plugs into existing PTQ pipelines with negligible overhead, and offers a complementary, easy-to-adopt lever for low-bit generation fidelity.

Post-training quantization for diffusion models increasingly exploits timestep, feature, and layer structure. While recent work has begun incorporating CFG structure into diffusion quantization, activation quantization still operates independently across conditional and unconditional coordinates, leaving cross-activation structure unexploited. We show that matched CFG activations form a strongly correlated two-dimensional source and that, under a fixed bit budget, the choice of branch coding basis materially affects quantization fidelity. Motivated by this observation, we introduce branch-space transform coding, which rotates matched CFG branches via an offline derived 2x2 orthogonal matrix, requiring minimal modifications to model parameters or the quantization pipeline. We further derive the Guidance-Correlation Branch Transform (GCBT), which jointly incorporates the CFG guidance direction and cross-branch second moments. Under an equal-rate quantization-noise surrogate, GCBT admits a closed-form per-layer solution without gradient optimization or angle search. Applied on top of existing diffusion PTQ methods, GCBT yields statistically significant fidelity gains in most evaluated comparisons with no statistically significant degradation, while leaving the underlying host quantization pipeline unchanged.
0
cs.CVcs.RO Hao Wu, Shengju Qian, Weiyan Wang et al. · Oct 1, 2026

FutureWorlds learns robotic world models by turning model-generated alternative futures into training experiences. The core idea unifies three ingredients: diverse beam search to build informative candidate trajectories, candidate-specific bounded memory to maintain diverging rollout histories, and MemSPO—a GRPO-based post-training algorithm that converts decoded trajectory rewards into group-relative advantages with history-consistent scoring. This addresses the exposure bias and token-level likelihood misalignment that plague autoregressive world models, where teacher-forcing training histories diverge from self-generated inference histories.

Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
0
cs.CV Junyi Wu, Fanqing Kong, Leyang Chen et al. · Oct 1, 2026

Feed-forward 3D reconstruction models like VGGT and $\pi^{3}$ rely on expensive quadratic global attention over long image sequences. VASC proposes a training-free sparse attention substitute that ranks block interactions by both query-key relevance and adjacent-value contrast, then tracks unserved computational demand across layers via a lightweight cross-layer memory. It achieves up to $2.29\times$ faster inference than dense VGGT while improving pose and reconstruction accuracy over prior sparse baselines.

Feed-forward 3D vision models such as VGGT have achieved remarkable progress, unifying camera estimation and dense scene reconstruction in a single pass. However, their quadratic global attention makes long image sequences expensive, while existing sparse methods may favor highly attended yet value-redundant regions. To address these limitations, we introduce VASC, a training-free sparse attention method combining value-aware block selection and execution-aware cross-layer memory. Our value-aware block selection integrates pooled query--key relevance with neighboring value contrast, reducing redundancy while preserving query-relevant and distinctive content. Cross-layer memory tracks unserved demand across layers and updates this state according to actual execution, enabling previously underserved blocks to compete under a fixed computation budget. Experiments on 7Scenes and NeuralRGB-D with VGGT and $\pi^3$ demonstrate improved pose estimation and reconstruction quality compared with FasterVGGT, together with up to $2.29\times$ faster inference than dense VGGT. Code is available at https://github.com/kosakayamahoo-design/VASC.
0
cs.CVcs.SDeess.AS Gunwoo Lee, Yoori Oh, Yoseob Han · Oct 1, 2026

This paper tackles video-to-speech (V2S) synthesis, where the central challenge is a one-to-many mapping: visually identical lip movements can correspond to different phonemes. The proposed Watch Your Speech (WYS) framework injects explicit textual conditioning—ground-truth text during training and lip-reader predictions at inference—into a conditional flow matching model via an attention-based fusion module. The key question is whether training-time textual guidance can outperform inference-only classifier guidance while preserving naturalness.

Video-to-speech synthesis aims to generate natural-sounding speech from silent talking-face videos while ensuring phonetic accuracy. A fundamental challenge in this task is the inherent one-to-many mapping problem, where visual dynamics often lack sufficient information to uniquely determine the corresponding utterance. To address this, we propose Watch Your Speech (WYS), a video-to-speech synthesis framework that incorporates textual conditioning as an explicit linguistic cue to mitigate visual ambiguity. Our framework features an attention-based embedding fusion module that synergistically integrates textual context with video sequences, coupled with a conditional flow matching objective for high-fidelity speech generation. Extensive experiments on the LRS2 and LRS3 datasets demonstrate that WYS achieves superior performance, establishing new state-of-the-art results in audio-visual synchronization (LSE-C/D) while maintaining highly competitive textual accuracy (WER). Subjective evaluations further confirm that our model generates speech with near-human naturalness, validating the effectiveness of textual conditioning in content-controlled video-to-speech synthesis. Project page: https://github.com/gunwoo5034/Watch-your-Speech
0
cs.CV Xianda Du, Max Ku, Weiming Ren et al. · Oct 1, 2026

VIEScore2 addresses the shortcoming that most automatic image evaluators return only a scalar quality score without identifying defective regions. It introduces a unified evaluator that represents an image as a sparse $16\times 16$ text grid and jointly predicts perceptual quality, semantic consistency, and defect cell coordinates in a single autoregressive pass. By framing localization as text generation, it enables directly verifiable post-training rewards and supports heterogeneous supervision across generation and editing tasks. The practical value lies in producing human-readable, spatially grounded feedback from one model without task-specific prediction heads.

Existing synthetic image evaluators typically provide only a scalar quality score and do not identify the image regions that support it. We introduce VIEScore2, a unified evaluator for image generation and editing tasks with optional conditioning images. VIEScore2 represents an image as an N x N grid and jointly predicts quality scores and defect locations in a single model pass. Its text-native grid representation provides a common interface for heterogeneous spatial supervision and enables directly verifiable post-training objectives. We train on 38K examples spanning score-only, localization-only, and joint supervision across generation and editing tasks. Starting from supervised fine-tuning, we further apply GRPO to improve defect localization using rewards that combine cell-level Dice overlap, score accuracy, and output-format validity. A parameter-free parser converts the structured predictions into readable explanations. On the primary suite, VIEScore2 achieves an overall-score SRCC of 0.601, compared with 0.491 for Gemini-3-Flash, the strongest zero-shot general-purpose VLM baseline under matched inputs. For defect localization, VIEScore2 outperforms both general-purpose VLMs and specialized spatial evaluators on three of six benchmarks in per-image grid IoU and ranks among the top three on five, including datasets beyond its training sources.
0
cs.LGcs.CVstat.ML Junghwan Park, Woojin Cho · Oct 1, 2026

The paper solves the problem that weight-space task arithmetic is trapped in one architecture because parameter updates share no common coordinate system. Its core idea is the platonic task vector—a model-agnostic object beneath per-model task vectors—and the Universal Task Descriptor (UTD), an architecture-independent matrix of fine-tuning-induced affinity shifts that can be added or negated directly. A closed-form operator folds into the target's last layer, while a low-rank adapter fits the same objective across all layers. The work matters because it enables label-free, weight-free composition of task knowledge across six heterogeneous vision–language and audio–text families.

Models specialized for the same task converge to similar behavior, yet the parameter updates that produce it share no common coordinate system, so weight-space task arithmetic stays confined to a single model and cannot cross architectures without a structural correspondence. Drawing on Plato's allegory of the cave, we hypothesize that these model-specific updates are shadows of one shared, model-agnostic object, which we call the platonic task vector. To make it operational for models that pair an image or audio encoder with a text encoder, we introduce Universal Task Descriptors: matrices whose shape is independent of architecture and embedding dimension, which record a task's functional effect and support addition and negation as matrix operations. Transferring a descriptor into a target means editing the target until it reproduces the descriptor on the task's unlabeled probe images and class-name prompts, requiring no per-image labels. We realize this edit in two ways. First, the descriptor factorizes into a shift field on image embeddings, so a single least-squares solve yields a linear operator that folds into the target's last layer as a weight edit; by linearity, a bank of such operators admits any composition at any strength as a signed sum. Second, a low-rank adapter trained on the same objective reaches every layer and fits compositions jointly, at the cost of one optimization per edit. Heterogeneous models share this object only partially, with a model-specific residual comparable in norm to the shared component, yet cross-model transfer still retains 74-80 percent of the gain of the target's own descriptors. Experiments across six model families, eight classification tasks, and an audio-text setting show that task knowledge transfers and composes across heterogeneous models under both realizations.
0
cs.ROcs.CVcs.LG Chengkai Xu, Yiming Cui, Jiaqi Liu et al. · Oct 1, 2026

End-to-end autonomous driving maps raw sensors directly to control actions, yet its real-world bottleneck lies in the training ecosystem rather than model architecture alone. This survey introduces a Data-Strategy-Platform taxonomy that frames training as an interdependent system: data defines what is learnable, strategies govern how learning aligns with driving objectives, and platforms enable scalability. The authors argue the field must shift from raw data volume to value density, from isolated task-specific learners to universal foundation models, and from static offline training to closed-loop training-testing integration.

Autonomous driving is a cornerstone technology for the future of intelligent transportation, where end-to-end learning has emerged as a transformative paradigm that directly maps multimodal sensory inputs to driving actions through unified differentiable models. While offering advantages, the effectiveness of end-to-end autonomous driving (E2E-AD) is ultimately determined by the quality of its training ecosystem. This paper provides a comprehensive review of training methods and ecosystem for E2E-AD. We introduce a Data-Strategy-Platform taxonomy that conceptualizes training as an interdependent system. The data layer defines what can be learned, the strategy layer governs how learning aligns with driving objectives, and the platform layer supports scalability and continuous evolution. Within this framework, we survey recent advances across data-centric pipelines, learning paradigms, and training infrastructures, and analyze their interplay in shaping model performance, robustness, and deployability. Finally, we reflect on current limitations and articulate a forward-looking vision that emphasizes a shift from data quantity to data value, from isolated optimization to foundation-driven generalization, and from static training to integrated training-testing loops, aiming toward robust, scalable, and trustworthy autonomous driving systems. We maintain a continuously updated repository tracking cutting-edge literature and works at \href{https://github.com/Jiaaqiliu/Awesome-Training-Ecosystem-for-E2E-AD}{Our Project Page}.
0
cs.CVcs.LG Jungmin Lee, Niamat Ullah, Yoseob Han · Oct 1, 2026

EyeTAG tackles temporal jitter in video gaze estimation by making gaze motion an explicit variable rather than an implicit deep feature. It feeds back first-order differences of its own recent predictions as a compact kinematic token, then fuses this with face and eye streams via cross-attention and a causal Transformer decoder. The result is a causal streaming model that cuts mean angular error on Gaze360 by roughly a degree and claims subject-invariant motion via translation-invariant differencing.

Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.
0
cs.LGcs.CV Sidney Bender, Benedikt Kunz, Ahmed Zeid et al. · Oct 1, 2026

Foundation models often rely on spurious correlations, yet generating visual counterfactual explanations is usually slow and entangled. This paper proposes DiDAE, a gradient-free method that wraps a frozen visual foundation model in a conditional diffusion decoder and performs closed-form edits along disentangled dictionary atoms. The approach promises up to 2000× speed-ups over iterative methods while enabling targeted model repair via Counterfactual Knowledge Distillation and unsupervised discovery of shortcut directions.

Foundation models remain vulnerable to spurious correlations and ``Clever Hans'' strategies. Explainable machine learning can find and remove such strategies for classifiers without metadata. For foundation models, no such option exists yet. We propose Disentangled Diffusion Autoencoders (DiDAE). DiDAE wraps a frozen foundation model in a conditional diffusion decoder. A counterfactual is one closed-form edit along a direction of a disentangled dictionary, followed by decoding. The dictionary can be supervised (Procrustes) or unsupervised (Singular Value Decomposition, Sparse Autoencoders). No gradients are needed, so DiDAE is up to 2000 times faster than the state of the art. We evaluate on six datasets, two synthetic and four real-world. In a desiderata-driven benchmark on three of them, its counterfactuals are on par with or better than the state of the art, and they repair downstream classifiers through Counterfactual Knowledge Distillation (CFKD), where they beat metadata-based correction. The same machinery can rank a pretrained dictionary against a trained classifier. It returns the few directions the classifier actually reads, each causally verified by a counterfactual that flips the decision, and repairs the classifier along those a teacher marks spurious. The workflow is plug-and-play in our open-source Peal library we publish alongside the paper. With a public dictionary and a pretrained decoder, all that remains is a cheap linear distillation of the classifier and its own fine-tuning.
0
cs.CV Ozge Mercanoglu Sincan, Anton Pelykh, Edward Fish et al. · Oct 1, 2026

Sign language machine translation remains far behind spoken-language MT because sign languages exploit simultaneous manual and non-manual channels, spatial grammar, and three-dimensional construction. This survey reviews the technical arc from isolated recognition to end-to-end translation and production, embedding every stage in sign language linguistics and deaf-community ethics. The authors argue that progress must be judged not by benchmark scores alone but by whether deaf users actually accept and understand the output, making the manuscript as much a piece of technology assessment as a call for sociotechnical reform.

Sign language machine translation has progressed substantially over the past decade, evolving from isolated sign recognition to end-to-end translation systems. Advances in pose estimation, transformer architectures, and large-scale dataset collection have driven progress, yet challenges remain. Datasets are limited compared to spoken-language resources; evaluation metrics inadequately capture the linguistic quality of output; and models must capture the simultaneous, multi-layered, and three-dimensional structure of sign languages. This manuscript provides a comprehensive review that seeks to balance technical challenges with stakeholder considerations. We examine the linguistic properties that make sign languages computationally unique, trace the evolution of recognition, translation, and production systems, and analyze ongoing technical challenges. Crucially, we address ethical considerations around data governance, community involvement, and appropriate use. Drawing on interdisciplinary perspectives spanning computer vision, sign language linguistics, and deaf studies, our analysis emphasizes that continued progress requires sustained collaboration across these fields and with deaf communities.
0
cs.ROcs.CVeess.IV Pengfei Qi, Haoran Lin, Sizhuang Chen et al. · Oct 1, 2026

UniTrackPLA tackles the problem of unifying instruction-guided navigation and dynamic person tracking—two embodied tasks traditionally handled by separate policies—within a single panoramic vision-language-action framework. Its core idea is to fuse omnidirectional perception with a shared continuous waypoint prediction space, using Panoramic-Aware Encoding to adapt perspective pretrained encoders to 360° observations and World-Action Consistency to adaptively decide when to reuse waypoint chunks or replan. This matters because it removes heading-dependent initialization constraints and offers a step toward general-purpose robots that can follow both linguistic routes and moving humans with one model.

General-purpose embodied robots should support both navigation toward language-specified destinations and dynamic person tracking under arbitrary initial target azimuths. However, existing methods typically rely on forward-facing observations and address these tasks with separate policies, limiting omnidirectional perception and unified closed-loop control. We present UniTrackPLA, a unified panorama-language-action model for instruction-guided navigation and dynamic person tracking. Its Panoramic-Aware Encoding (PAE) preserves the temporal and azimuthal structure of perspective views projected from each panorama, enabling perspective-pretrained visual encoders to process omnidirectional observations. A shared vision-language backbone grounds instructions in the panoramic context and predicts continuous robot-centric waypoint chunks for both tasks. World-Action Consistency (WAC) further predicts action-conditioned future visual states and verifies waypoint prefixes online, allowing reliable actions to be reused while triggering replanning upon inconsistency. We also introduce OmniTrackNav-Bench, comprising 5,000 simulated tracking trajectories, 10,000 simulated VLN routes, and 96 verified real-world routes, providing 919,978 waypoint-supervision instances. UniTrackPLA improves overall tracking SR from 23.50% to 35.00% and Omni-VLN SR/SPL from 13.00%/12.77% to 19.75%/19.29%. Incorporating 76 real-world routes further improves held-out EP@0.2m from 42.92% to 92.08%. Closed-loop experiments on a Go2-W robot demonstrate unified panoramic tracking and navigation across indoor and outdoor environments. The project page is at https://tw5775.github.io/UniTrackPLA.
0
cs.ROcs.AIcs.CL Jiawei Fan, Sifeng Wang, Yuqing Hou et al. · Oct 1, 2026

Robotic foundation models currently rely on multi-step flow matching to generate actions, which creates a severe inference bottleneck for real-time control. This paper proposes Kinematic MeanFlow (K-MF), a one-step action policy that reformulates MeanFlow by decoupling the time derivative across two sub-intervals to accommodate the unique late-stage surge and widening spread of local acceleration in robotic velocity fields. By introducing an intermediate anchor point and a learnable partition strategy, K-MF cuts action-head latency by up to 74% while matching or exceeding the success rates of multi-step baselines across manipulation and navigation tasks.

In this paper, we study how to achieve one-step action generation in Robotic Foundation Models (RFMs), aiming to overcome the high inference latency of multi-step flow matching. MeanFlow provides a promising framework for this goal, yet its direct application leads to performance collapse. We discover that this stems from two distinctive dynamics exhibited in the RFM velocity field: (1) the ``local acceleration" exhibits stability early on, but surges sharply towards the end of the denoising process, and (2) the spread of its magnitudes across samples widens as denoising progresses. To address these issues, we introduce Kinematic MeanFlow (K-MF), a novel one-step action policy tailored for RFMs. Specifically, grounded in a kinematic identity, K-MF decouples the time derivative term in the MeanFlow formulation into two sub-interval terms separated by an intermediate point. This decoupled formulation enables the two terms to capture early-stage and late-stage denoising dynamics, respectively, while mitigating the error amplification across the process. As a result, our K-MF empowers RFMs to achieve one-step action generation in both training from scratch and fine-tuning paradigms across diverse tasks, while outperforming multi-step flow matching in most settings. In terms of inference efficiency, K-MF reduces action-head latency of GR00T-N1.6 by 67.5%~74.4% across L40 and Jetson Orin in eager and compiled modes, yielding end-to-end latency reductions of 30.3%~54.9%. Code will be available at https://github.com/IntelChina-AI/K-MF.
0
cs.LGcs.AIcs.CV Melika Shirian, Kianoosh Vadaei · Oct 1, 2026

This paper studies adversarial attacks under a shared global mean-$\ell_1$ budget with per-coordinate magnitude caps. The core idea is to add a centered, clean-gradient importance prior to the attack objective to softly guide perturbation mass toward model-sensitive regions without changing the feasible set. This matters because, under local capacity constraints, simply finding a useful gradient direction is insufficient—attackers must also decide where to spend a budget that cannot be arbitrarily concentrated. Empirically, the mechanism raises attack success by 2.52–17.70 percentage points over matched baselines across ten robust model–dataset configurations.

Adversarial optimization under a shared $\ell_1$ budget requires deciding not only how much perturbation to use, but also where that limited budget should be spent. This allocation problem becomes particularly important when individual input coordinates are subject to local magnitude constraints, which restrict the extent to which perturbation can be concentrated on a small number of locations. We introduce an importance-guided allocation mechanism that uses a fixed clean-gradient prior to steer perturbation toward model-sensitive regions while leaving the feasible perturbation set unchanged. A centered allocation objective encourages perturbation at above-average importance locations and discourages unnecessary expenditure elsewhere, thereby redistributing rather than enlarging the available budget. Across ten robust model--dataset configurations under a common capacity-limited threat setting, the proposed method improves attack success over matched APGD- and PMA-based baselines by $2.52$ to $17.70$ percentage points. Allocation analysis shows that these gains are accompanied by substantially greater perturbation mass in high-importance regions without increased global $\ell_1$ consumption. Mechanism ablations further show that centered non-uniform redistribution provides part of the benefit, while model-derived importance yields an additional improvement. These results identify perturbation allocation as a distinct and practically relevant dimension of adversarial optimization under shared-budget, locally constrained threat models.
0
eess.IVcs.CVq-bio.QM Song Zhiying, Ling Hanyi, Wu Junyi et al. · Oct 1, 2026

MorphoBranch tackles the problem of accurately quantifying fine, weakly labeled, and densely branched cellular structures—such as neuronal axons and microglial processes—from fluorescence microscopy images. The core idea is an integrated workbench that pairs a deterministic image-to-graph Morphometry Engine (multiscale ridge evidence, hysteresis segmentation, and evidence-constrained skeleton refinement) with an optional LLM-assisted Refinement Engine that translates natural-language instructions into editable, human-reviewable analysis plans. This separation of flexible interaction from deterministic computation aims to reduce operator-dependent bias while keeping morphometric workflows inspectable and reproducible.

Background and Objectives: Fluorescence-labeled cellular arbors provide readouts of neuronal and microglial morphology, but fine and weakly labeled processes are prone to fragmentation and false connections that bias skeleton-based measurements. We present MorphoBranch, a fine-structure-preserving, human-reviewable workbench for morphometry of branched cellular structures. Methods: MorphoBranch combines a deterministic Morphometry Engine with an LLM-assisted Refinement Engine. The Mor- phometry Engine implements an image-to-graph workflow integrating multiscale structural evidence extraction, hysteresis segmen- tation, evidence-constrained skeleton refinement, and graph-based morphometry. The Refinement Engine maps natural-language requests to registered actions for parameter adjustment, preview execution, metric reporting, and unsupported-request handling, while image processing and quantitative computation remain deterministic and reviewable. Results: MorphoBranch was evaluated on two public neuronal axon datasets, AxonMIP and AxonStack, and the in-house Cell- Morph dataset of microglial fluorescence images. It achieved the highest Skeleton F1 and clDice and the lowest length-estimation error among the evaluated methods on all three datasets, while also achieving the highest Dice and IoU on AxonMIP and Axon- Stack. Across 150 natural-language tasks, the Refinement Engine achieved a 94.0% end-to-end success rate. Conclusions: These results demonstrate that MorphoBranch provides a reproducible, human-reviewable workflow for mor- phometric analysis of branched cellular structures. It supports fine-structure-preserving quantification across neuronal axon and microglial fluorescence images while maintaining inspectable and reproducible analysis workflows.
0
cs.CV Chensheng Peng, Wenhao Ding, Ran Tian et al. · Oct 1, 2026

World Action Models jointly predict future actions and video, but standard training independently corrupts actions and video with noise, creating a physical mismatch: perturbed actions imply a different visual future, yet the noised video stays tied to the original recording. CtrlWAM fixes this by executing perturbed actions in a simulator and pairing the rendered consequences with the perturbed actions for training, augmented with warped video–action noise schedules and a structured multi-agent conditioning interface. The approach targets more physically consistent and controllable world models for autonomous driving and robotic manipulation.

World action models (WAMs) jointly predict actions (intent) and visual future (foresight). Standard training adds noise to recorded actions and video simultaneously, but such training paradigms introduce a mismatch: perturbed actions imply counterfactual future visual, while the noised video remains tied to the GT recording. In low-noise regime, the scene geometry and even the dynamic behavior remain clearly visible from the noisy future frames despite the added noise. We present CtrlWAM, which executes perturbed actions in a simulator and pairs them with their noised visual consequences for joint WAM learning. To accommodate the different denoising requirements of video and actions, we introduce warped video--action noise schedules that aim to keep visual layout responsive as action predictions evolve. We further extend the action interface from ego-only control to a variable number of agent streams, allowing a unified model to represent predicted or commanded futures for multiple agents. Driving experiments show more accurate action forecasts, closer agreement between generated video and actions, and better following of supplied commands; robotics experiments show stronger motion fidelity and controllability. Matched controls support the benefit of off-path renders for command following and manipulation fidelity. Together, these findings contribute to a more controllable world action model. Project page: https://ctrl-wam.github.io/
0
cs.CVcs.RO Cigdem Kokenoz, Amir Salarpour, Alkim Domeke et al. · Oct 1, 2026

Lang3DSeg tackles annotation-free open-vocabulary 3D segmentation of sparse outdoor LiDAR using a point transformer backbone trained from scratch. Its core idea is to carefully curate 2D-to-3D pseudo-labels via class-priority rasterization and an instance-wise depth-gap test that truncates projected masks at the first occlusion gap, removing systematic depth-ambiguity errors before training. This matters because prior methods rely on voxel-based backbones and temporal voting to compensate for projection noise, whereas Lang3DSeg shows that a point transformer can match and exceed those results in real time when paired with cleaner single-sweep targets.

Accurate 3D semantic perception is critical for safe autonomous navigation. However, supervised LiDAR segmentation remains tied to closed taxonomies and to the cost of point-wise manual annotation. Open-vocabulary methods avoid that cost by projecting the output of 2D vision-language models onto LiDAR and distilling it into a 3D network. These methods rely almost exclusively on voxel-based sparse convolutions, and point transformers have so far been limited to indoor environments, where 3D data is dense and bounded. We present Lang3DSeg, which establishes a point transformer as the backbone for annotation-free open-vocabulary segmentation of outdoor 3D LiDAR, and is trained from scratch without geometric pre-training. This training paradigm necessitates addressing the inherent noise in 2D-to-3D label projections; specifically, naive projection often suffers from depth ambiguity, where points behind an object are erroneously assigned its semantic label. We therefore composite masks using an explicit class-priority rule and truncate each projected instance at the first gap in its depth distribution, correcting the projection error directly rather than averaging it over registered sequences. Lang3DSeg achieves 52.8% mIoU on nuScenes validation and 41.4% on SemanticKITTI, the highest among published annotation-free methods on both benchmarks. Every 3D semantic segmentation is on a single LiDAR sweep, and inference operates in real-time without running vision-language models.
0
cs.CVcs.LG Thiru Thillai Nadarasar Bahavan, Yu Xia, Sachith Seneviratne et al. · Oct 1, 2026

Open Set Recognition (OSR) requires models to classify known classes accurately while rejecting samples from unseen categories, which depends critically on learning compact, well-separated feature clusters. This paper proposes SmoothOperator (SmoothOP), a plug-in that adaptively sets the label smoothing coefficient for each training sample based on its prominence—an embedding-space signal measuring how clearly a sample's own class dominates its strongest competitor. By modulating smoothing sample-by-sample, SmoothOP yields tighter and better dispersed class representations, improving OSR performance across fine-grained benchmarks.

Open Set Recognition (OSR) aims to enable models to accurately classify known classes while rejecting samples from unseen classes. A key challenge in OSR lies in the inability to model the unbounded distribution of unknown classes during training, often leading to the misclassification of samples from these classes. Rather than modeling unknowns, recent work shapes the feature space so that known classes are compact and well separated, and spherical representation learning methods have achieved strong results this way. Label smoothing has been identified as one of the key drivers of this success, yet it applies the same coefficient to every training sample, regardless of how well each sample is already embedded. We show that the spherical representation learning objectives used in OSR share a single alignment--uniformity structure in which labels enter only through the alignment term. Label smoothing therefore acts as an alignment dial, and a fixed coefficient sets this dial to the same value for every sample. We propose a plug-in, SmoothOperator (SmoothOP), which sets the smoothing coefficient of each sample from its \textbf{prominence}, an embedding-space signal measuring how clearly the sample's own class stands out against its strongest competing class. Our method integrates into four existing spherical representation learning methods at minimal training overhead. SmoothOP assigns strong smoothing to samples with high prominence, which reduces their alignment and relaxes their pull. On the Semantic Shift Benchmark, SmoothOP-augmented variants generally outperform their base objectives across datasets, degrees of semantic shift, and OSR post-processors, with gains of up to 4.7\% in AUROC, OSCR, and closed-set accuracy.
0
cs.CVcs.AI Shao-Jun Xia, Huixin Zhang, Zhen Lei et al. · Oct 1, 2026

Vision-language models are increasingly used as backbones for all-in-one low-level image restoration, yet how they internally organize pixel-level perceptual representations remains unclear. This paper introduces GeoSim, a unified four-level framework that probes global similarity, local geometry, sparse feature decomposition, and topological structure across 24 tasks in autoregressive and diffusion transformer VLMs. By revealing that task representations are mechanism-centric and that cross-model agreement is primarily global rather than local, the study offers an actionable interpretability lens for designing transferable unified restoration systems.

Vision-language models (VLMs) have emerged as powerful candidates for universal vision backbones, with representative architectures including autoregressive (AR) models and diffusion transformers (DiTs). Yet, adapting them efficiently for all-in-one low-level image restoration remains a challenge. Crucially, the field lacks an understanding of how VLMs organize hidden-layer representations and whether these structurally distinct paradigms share a common geometric organization for pixel-level perception. Such shared organization is a prerequisite for building highly transferable, unified restoration VLMs and adapters. In this paper, we systematically investigate representational similarity across 24 low-level tasks spanning 5 categories. We propose GeoSim, a unified four-level framework that analyzes task-conditioned representations from global similarity, local geometry, sparse feature decomposition, and topological verification perspectives. Our formulation applies to the analysis of hidden states in AR models and feature maps in DiTs across same- and cross-task/model settings. Our results reveal the organizing principles of low-level visual representations while exposing their limits in cross-task and cross-model agreement. Ultimately, GeoSim provides an interpretability lens for probing latent transferability in low-level vision and diagnosing model limitations in task- or model-specific scenarios.
0
cs.ROcs.HCcs.RO Thomas M. Kwok, Jiaan Li, Yue Hu · Apr 9, 2025

Remote caregiving robots could help older adults age in place, but conventional motion-mapping teleoperation forces caregivers into slow, unnatural postures that cause muscle fatigue. This paper proposes a lightweight Spatio-Temporal Graph Convolutional Network (S-ST-GCN) that recognizes four single-arm daily-living actions from an RGB camera and triggers preset robot trajectories, decoupling the operator's motion from the robot's execution. A finite-state machine filters misclassifications before execution. While still a proof-of-concept, the framework offers a more intuitive alternative to direct motion-tracking teleoperation for assisting with activities of daily living.

Caregiving of older adults is an urgent global challenge, with many older adults preferring to age in place rather than enter residential care. However, providing adequate home-based assistance remains difficult, particularly in geographically vast regions. Teleoperated robots offer a promising solution, but conventional motion-mapping teleoperation imposes unnatural movement constraints on operators, leading to muscle fatigue and reduced usability. This paper presents a novel teleoperation framework that leverages action recognition to enable intuitive remote robot control. Using our simplified Spatio-Temporal Graph Convolutional Network (S-ST-GCN), the system recognizes human actions and executes corresponding preset robot trajectories, eliminating the need for direct motion synchronization. A finite-state machine (FSM) is integrated to enhance reliability by filtering out misclassified actions. Our experiments demonstrate that the proposed framework enables effortless operator movement while ensuring accurate robot execution. This proof-of-concept study highlights the potential of teleoperation with action recognition for enabling caregivers to remotely assist older adults during activities of daily living (ADLs). Future work will focus on improving the S-ST-GCN's recognition accuracy and generalization, integrating advanced motion planning techniques to further enhance robotic autonomy in older adult care, and conducting a user study to evaluate the system's telepresence and ease of control.
0
cs.LGcs.LG Huanran Chen, Huaqing Zhang, Xiao Li et al. · Apr 10, 2026

This paper investigates the geometric structure of converged states in LLM pretraining, asking whether models converge to a common minimizer across data sources or merely a minimizer of the summed loss. The authors hypothesize that the "closeness" of task-specific minima correlates with downstream generalization, and propose the Nexus optimizer to maximize gradient similarity as a tractable proxy for closeness. Their core finding—that identical pretraining loss can mask vastly different downstream performance depending on the implicit bias toward geometric closeness—challenges the prevailing reliance on pretraining loss as the sole evaluation metric.

Pretraining is the cornerstone of Large Language Models (LLMs), dominating the vast majority of computational budget and data to serve as the primary engine for their capabilities. During pretraining, LLMs acquire foundational knowledge from an unprecedentedly massive and diverse data sources, encompassing a vast array of domains such as general language, mathematics, code, and complex reasoning. In this work, we investigate an interesting geometric question regarding the converged state of pretraining: Does the model converge to a common minimizer across all data sources (e.g., \cref{fig:cwa_illustration:close}), or merely a minimizer of the summed loss (e.g., \cref{fig:cwa_illustration:distant})? We hypothesize that the geometric "closeness" of task-specific minima is intrinsically linked to downstream generalization. We reveal that standard optimizers (e.g., AdamW) often converge to points where task-specific minima are distant from each other. To address this, we propose the Nexus optimizer, which encourages the closeness of these minima by maximizing gradient similarity during optimization. Experiments across models ranging from 130M to 3B parameters, various data mixtures and hyperparameter schedules, show that Nexus \textit{significantly boosts downstream performance}, despite \textit{achieving the same pretraining loss} (see \cref{fig:demo:benchmark}). Notably, on the 3B model, Nexus reduces the out-of-distribution loss by 0.012 and yields up to a 15.0\% accuracy improvement on complex reasoning tasks (e.g., GSM8k). This finding challenges the reliance on pretraining loss as the sole proxy for model evaluation and demonstrates the importance of implicit biases in unlocking downstream generalization.