Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
398 papers in cs.CV
Trending mixes fresh papers with community signal.
0
cs.CV Sejin Park, Hongjae Lee, Changwoo Han et al. · Oct 1, 2026

Logic gate networks replace arithmetic operations with purely Boolean functions for highly efficient inference, yet scaling their width naively causes kernel redundancy and performance saturation. This work identifies that redundancy is most severe at the first gate level and proposes Dynamic Logic Kernel (DLK) and Early-stage Dynamic Logic Kernel (EDLK), which use input-dependent Boolean routing to specialize kernel groups or first-level gate groups. The result is improved kernel diversity and better accuracy-efficiency trade-offs under width scaling, advancing the practicality of discrete logic-based deep learning.

Differentiable logic gate networks, which operate using only logic gates, have recently attracted attention as an efficient alternative to conventional neural networks. However, despite their efficiency, the scaling behavior of logic gate networks remains underexplored. By contrast, scaling model capacity is a central design principle in deep neural networks and typically leads to improved performance. This discrepancy raises a key question: Can similar scaling benefits also be achieved in logic gate networks? In this work, we focus on width as a primary scaling axis and conduct a systematic analysis of its behavior in logic gate networks. We observe that naive width scaling often introduces redundancy among logic kernels, limiting the effective use of additional kernels and leading to performance saturation. To address this limitation, we propose a dynamic logic kernel framework that reorganizes kernel utilization by promoting specialization across kernel groups. This enables the network to better utilize increased width via input-dependent kernel routing, while ensuring that both routing and computation are implemented entirely with gate-level Boolean operations at inference time. We further find that kernel redundancy is most pronounced at the first gate level, motivating an early-stage dynamic logic kernel strategy that concentrates adaptation at this level. Experimental results demonstrate that our approach improves kernel utilization and increases kernel diversity, leading to higher accuracy with improved parameter efficiency.
0
cs.CVcs.AI Minsu Kim, Jaesung Choe, Jiwoo Lee et al. · Oct 1, 2026

RelationVGGT tackles 3D spatial relation segmentation: given multi-view images, a reference subject mask, and a text relation (e.g., "resting on"), it segments the target object across unposed views without per-scene optimization. The core idea is to replace expensive volumetric relation search with per-pixel relation prediction by fusing semantic DINOv2 features with geometry-aware Pi3 tokens through a subject-conditioned relation transformer. This matters because it moves holistic scene understanding beyond isolated object recognition toward reasoning about how objects are spatially arranged, a capability critical for embodied agents and spatial QA.

Recent advances in 3D reconstruction have progressed from per-scene optimization to feed-forward inference, and semantic scene understanding has followed suit -- yet existing methods remain confined to object-centric perception, neglecting spatial relations between objects. We formulate 3D spatial relation segmentation in a feed-forward, pose-free multi-view setting: given a visually specified subject and a relational text query, the model segments the target across views without receiving its category name. To this end, we propose RelationVGGT, a novel feed-forward framework that integrates semantic features from a visual foundation model with geometry-aware representations from a 3D geometry foundation model and leverages a relation transformer for subject-conditioned, cross-view relation prediction -- requiring neither per-scene optimization nor known camera poses. We additionally provide a fully automated annotation pipeline built on ScanNet++ with VLMs and LLMs, enabling scalable training data generation for this new task.
0
cs.CVeess.IV Bi'an Du, Zhimin Zhang, Daizong Liu et al. · Oct 1, 2026

Sparse-view 3D reconstruction from only a handful of images is fundamentally ill-posed: standard methods overfit to individual views and leave unseen regions as holes or floaters. HierGF tackles this with a hierarchical 3D Gaussian Splatting pipeline that separates a coarse geometric backbone from a fine-scale field which inherits only spatial means and colors while re-estimating shape and appearance under enhanced pseudo-supervision. Geometry-consistent densification and confidence-weighted diffusion priors are then used to fill gaps and suppress artifacts in under-sampled regions. The authors report strong gains on MipNeRF360, OmniObject3D, LLFF and DTU, alongside a sub-11-minute per-scene reconstruction time on an A100 GPU.

Sparse view 3D reconstruction is an important and common scenario in multimedia applications, such as augmented reality/virtual reality (AR/VR) content creation, cultural heritage digitization, and certain robotic applications, where only a limited number of randomly captured views may be available. However, sparse views contain only limited 3D information, posing two major challenges:1) too few images are available for matching, making it difficult to build multi-view consistency; 2) insufficient view coverage leads to a lack of information in under-sampled regions, resulting in missing parts of object structure. Existing methods mostly still rely on limited reprojection errors and regularization terms, which are prone to overfitting to a single view and inconsistent appearances across views. In geometrically under-sampled regions, they often rely on heuristic density control, lacking reliable guidance and often resulting in blurring and structural holes.To address these issues, this paper proposes Hierarchical Gaussian Fields (HierGF), which revisits sparse-view reconstruction from a hierarchical geometry-perception perspective and converts limited observations into reliable self-generated supervision beyond fixed priors and heuristic density control. In particular, we transform coarse 3D geometric information and additional 2D generative priors into structured pseudo-supervision through a two-stage geometry-perception backbone network, thereby enhancing multi-view consistency with very few input views. In addition, we introduce a learnable confidence network to guide gradients toward cross-view consistent content, and a geometrically consistent densification module to improve the reconstruction of multi-view alignment and under-sampled regions.
0
cs.CV Enxin Song, Yinuo Xu, Shusheng Yang et al. · Oct 1, 2026

Video benchmarks are supposed to certify that models can understand temporal sequences, yet most can be gamed without watching the video. This paper audits 115 multiple-choice video benchmarks with a five-level attack pyramid that grants attackers access to answer options, question text, other items, single frames, or shuffled videos. After finding severe shortcut vulnerabilities—35 benchmarks break without any visual input and near-duplicates plague 63 of them—the authors screen half a million items down to 840 hardest survivors and compose them into Video-Index, a reproducible meta-benchmark backed by agent-assisted selection and lockfiles.

A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. We introduce the attack pyramid, five levels of shortcut attacks with increasing access to each item, and audit 115 video benchmarks with it. On 35 benchmarks, attackers that never see a frame approach full-video accuracy. On 51 benchmarks with temporal probes, shuffled frames keep a median 96% of full-video accuracy. Near-duplicate questions make up at least half the items in 63 benchmarks. We screen 505,518 question-answer pairs from 112 of them into an audited pool. Agents turn evaluation requests into specifications, and a deterministic selector with a red-team gate composes reproducible benchmarks. We release Video-Index, the 210 hardest verified items under these attacks in each of four capability groups, 840 items from 76 sources. With the same fixed input, Claude Opus 5 outscores every open-source model by over 37 percentage points, and agent tools add about 20 more, yet all systems leave room to improve efficiency and accuracy. Blog: https://www.enxinsong.com/blog/video-index/ GitHub: https://github.com/Espere-1119-Song/Video-Index Hugging Face: https://huggingface.co/datasets/Video-Index/Video-Index
0
cs.CV Tongcheng Zhang, Jun Zhu, Jianfei Chen · Oct 1, 2026

As video generation scales to minute-long clips, subjects enter and leave the scene, making static-set consistency metrics inadequate. This paper introduces DynSC-Eval, a framework that tracks each subject over its entire visible lifespan and decomposes consistency into local continuity and global identity preservation via six object-level metrics. The authors further derive rewards from these metrics and use DiffusionNFT to post-train video models, demonstrating substantial consistency gains in an autonomous driving testbed and extending them to 30-second generation through curriculum learning.

We argue that as video generation extends to longer durations, subject consistency should be evaluated over \textit{dynamic subject sets}. We therefore introduce \textbf{DynSC-Eval}, an evaluation framework that dynamically tracks eligible subjects throughout their visible lifespans and measures local continuity and global identity preservation using six complementary object-level metrics, with explicit detection of inconsistency events. To validate its effectiveness, we design synthetic experiments that actively inject inconsistency events, demonstrating both the sensitivity of DynSC-Eval and the limitations of existing metrics. Evaluations of diverse models on 5s, 15s, and 60s video generation further reveal substantial subject consistency differences that are obscured by conventional metrics. Beyond evaluation, we construct rewards from DynSC-Eval and apply DiffusionNFT post-training in an autonomous-driving testbed. On 5s generation, our approach reduces the six inconsistency metrics by an average of 13.82\% for Wan-2.1-1.3B and 5.66\% for SANA-2B, with improvements also observed on the I2V model ReSim. Qualitative comparisons further demonstrate the effectiveness of our method. We then extend generation to 10s and 30s through curriculum learning and show that consistency optimization remains effective while largely preserving other capabilities.
0
cs.CVcs.AI Keuntae Kim, Yong Suk Choi · Oct 1, 2026

This paper studies masked diffusion multimodal language models and shows that answer candidates often stabilize retrospectively long before their positions are permanently committed or the rationale-side canvas is filled. The authors introduce two clocks—token stabilization $T_{\mathrm{stab}}$ and commitment $T_{\mathrm{commit}}$—and measure how much of the rationale region $\mathcal{R}$ remains unwritten at stabilization. They pair this with matched-canvas visual-sensitivity ablations and a coverage-accuracy decomposition to explain why prompting changes affect overall accuracy. The analysis matters because it reveals that observing the final rationale alone is insufficient to understand when or how a diffusion model decides its answer.

An answer candidate in a masked diffusion MLLM can stabilize while its rationale is still unfolding. We distinguish retrospective stabilization of the logged candidate from token commitment, and examine these two clocks relative to rationale generation. Analyzing our results across three visual question-answering benchmarks, we find that 89.4-98.1% of the rationale-side canvas remains unwritten at stabilization in single-block, EOS-suppressed LaViDa runs. On V*Bench, reducing block length from 128 to 8 changes this fraction from 89.4% to 1.7%, together with answer coverage and the eligible observation window. Under EOS-enabled prompting, direct instructions improve Nemotron's overall accuracy by 15.0 and 19.5 percentage points on M3CoT and ScienceQA, but reduce LaViDa/V*Bench accuracy by 11.0 points. A symmetric decomposition associates the larger absolute component of each change with coverage rather than conditional accuracy. Matched-canvas image ablations measure visual sensitivity alongside answer stabilization, separating the two temporal readouts. Together, these measurements distinguish answer stabilization, rationale unfolding, and visual sensitivity, and identify coverage as the larger component of the prompting differences.
0
cs.CVcs.AI Giyeong Oh, Junghun Park, Yuhan Bae et al. · Oct 1, 2026

This paper attacks a growing bottleneck in text-to-image training: how to audit billion-scale recaptioned corpora before spending millions of GPU-hours on downstream training. It proposes a matched-budget audit framework that fixes a text budget $B=64$ and profiles any supervision distribution $D_{\pi,V_c,C}$ across five axes—prompt-side coverage, image-conditioned faithfulness, and caption-surface health—using controllable basic units (CBUs) as a common claim currency. The work matters because it disentangles corpus quality from model-training confounders and releases both a ~490M-caption corpus and a reusable artifact bundle.

Recaptioned image-text corpora are now standard for text-to-image (T2I) training, with vision--language model (VLM) captioners replacing sparse alt-text by dense descriptions. A recaptioned corpus is a supervision distribution induced by a documented captioning policy ($\pi$), captioner ($V_c$), and source corpus ($C$). Length-correlated proxies miss caption-register artifacts and downstream T2I benchmarks entangle the corpus with training choices, so this distribution is hard to audit at corpus scale. We introduce a reusable matched-budget audit framework for recaptioned supervision distributions $D_{\pi,V_c,C}$: at a fixed text budget of $B = 64$ it reports a five-axis profile spanning prompt-side coverage, image-conditioned faithfulness, and caption-surface health, with claimed controllable basic units (CBU) as the common claim unit. We instantiate the framework on seven paired comparisons over five public source corpora. Across the four cross-corpus pairs, the released surface raises supported CBU per caption by $+3.39$ to $+6.36$ under both Qwen and Gemma Judges, and on CC12M the same framework exposes a long-vs-dense frontier that is consistent across both judges and four budgets. We release the audited multi-source recap corpus ($\approx$ 490M) together with the audit-artifact bundle.
0
cs.CV Jiho Jang, Jinyoung Kim, Nojun Kwak et al. · Oct 1, 2026

Recent video generative models produce stunning visuals but routinely fail at plausible physical interactions and state transitions. This paper proposes a modular synthetic-data pipeline that couples a structured taxonomy with image-editing-based start/end anchors and a novel State-Guided Sampling (SGS) method to smoothly interpolate between them. Fine-tuning a Wan 2.1 model on this curated dataset improves interaction realism, backed by a hybrid VLM-plus-probe evaluator validated against human judgments.

While recent video generative models can synthesize high-fidelity videos, they struggle to portray plausible physical interactions and the resulting state transitions, a critical bottleneck for applications in robotics and VR/AR. To address this, we introduce a framework to generate a scalable synthetic dataset of controllable interactions. Our pipeline leverages a structured taxonomy and state-of-the-art image editing models to create explicit `start' and `end' state images, which serve as visual anchors for the interaction. To generate a seamless video utilizing these anchors, we propose State-Guided Sampling (SGS), a novel sampling technique that mitigates artifacts common in naive conditional generation. Furthermore, we develop and validate a new automated evaluation system that aligns with human judgments to ensure data quality. Experiments show that fine-tuning a base model on our dataset significantly enhances its ability to generate plausible interactions.
0
cs.CV Arash Rocky, Q. M. Jonathan Wu · Oct 1, 2026

SAM3-ASH tackles fully automated video tracking annotation without any dataset-specific training. Its Generalized Presence Token (GPT) enables SAM3 to process multiple text prompts simultaneously by virtual batching over a single shared image encoding, collapsing per-prompt cost from $\mathcal{O}(N)$ to $\mathcal{O}(1)$. The Annotation and Segmentation Handler (ASH) splits videos of arbitrary length into overlapping temporal chunks and reconciles identities across boundaries via IoU matching, bounding peak GPU memory below 25 GB. Evaluated on eight benchmarks, the pipeline sets a zero-shot HOTA record on MOTS20 and stays within a narrow gap of trained specialists on tasks where mask-based output aligns with the ground-truth format.

Memory-attention-based Video Instance Segmentation (VIS) methods have demonstrated strong zero-shot tracking capability, yet their substantial memory requirements confine them to short video clips and their single-prompt inference design makes multi-category open-vocabulary tracking computationally prohibitive. This work introduces two contributions toward fully automated tracking annotation of arbitrary video. The Generalized Presence Token (GPT) reformulates SAM3's inference pipeline to process N text prompts simultaneously via virtual prompt batching, reducing image encoding cost from O(N) to O(1) with no modifications to any learned component. The Annotation and Segmentation Handler (ASH) extends any memory-attention VIS tracker to sequences of arbitrary length through overlapping temporal chunks with IoU-based inter-chunk identity matching, requiring no dataset-specific training. Instantiated on SAM3, the resulting pipeline -- SAM3-ASH -- achieves state-of-the-art HOTA on MOTS20 under fully zero-shot conditions and remains competitive with trained specialists across seven additional benchmarks, while peak GPU memory consumption stays below 25 GB, establishing a practical baseline for scalable, training-free automated video annotation.
0
cs.CVstat.ML Mingrun Jiang, Yuejia Liu, Zishan Shao et al. · Oct 1, 2026

Diffusion models with classifier-free guidance (CFG) compute paired conditional and unconditional activations at each denoising step, yet existing post-training quantizers treat those branches as independent sources. This paper identifies the CFG branch space as a structured coding axis—matched activations are strongly correlated and jointly shaped by the guidance direction—and proposes Guidance-Correlation Branch Transform (GCBT), a lightweight $2\times 2$ orthogonal rotation applied per layer before activation quantization. It admits a closed-form per-layer solution, plugs into existing PTQ pipelines with negligible overhead, and offers a complementary, easy-to-adopt lever for low-bit generation fidelity.

Post-training quantization for diffusion models increasingly exploits timestep, feature, and layer structure. While recent work has begun incorporating CFG structure into diffusion quantization, activation quantization still operates independently across conditional and unconditional coordinates, leaving cross-activation structure unexploited. We show that matched CFG activations form a strongly correlated two-dimensional source and that, under a fixed bit budget, the choice of branch coding basis materially affects quantization fidelity. Motivated by this observation, we introduce branch-space transform coding, which rotates matched CFG branches via an offline derived 2x2 orthogonal matrix, requiring minimal modifications to model parameters or the quantization pipeline. We further derive the Guidance-Correlation Branch Transform (GCBT), which jointly incorporates the CFG guidance direction and cross-branch second moments. Under an equal-rate quantization-noise surrogate, GCBT admits a closed-form per-layer solution without gradient optimization or angle search. Applied on top of existing diffusion PTQ methods, GCBT yields statistically significant fidelity gains in most evaluated comparisons with no statistically significant degradation, while leaving the underlying host quantization pipeline unchanged.
0
cs.CVcs.RO Hao Wu, Shengju Qian, Weiyan Wang et al. · Oct 1, 2026

FutureWorlds learns robotic world models by turning model-generated alternative futures into training experiences. The core idea unifies three ingredients: diverse beam search to build informative candidate trajectories, candidate-specific bounded memory to maintain diverging rollout histories, and MemSPO—a GRPO-based post-training algorithm that converts decoded trajectory rewards into group-relative advantages with history-consistent scoring. This addresses the exposure bias and token-level likelihood misalignment that plague autoregressive world models, where teacher-forcing training histories diverge from self-generated inference histories.

Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
0
cs.CV Junyi Wu, Fanqing Kong, Leyang Chen et al. · Oct 1, 2026

Feed-forward 3D reconstruction models like VGGT and $\pi^{3}$ rely on expensive quadratic global attention over long image sequences. VASC proposes a training-free sparse attention substitute that ranks block interactions by both query-key relevance and adjacent-value contrast, then tracks unserved computational demand across layers via a lightweight cross-layer memory. It achieves up to $2.29\times$ faster inference than dense VGGT while improving pose and reconstruction accuracy over prior sparse baselines.

Feed-forward 3D vision models such as VGGT have achieved remarkable progress, unifying camera estimation and dense scene reconstruction in a single pass. However, their quadratic global attention makes long image sequences expensive, while existing sparse methods may favor highly attended yet value-redundant regions. To address these limitations, we introduce VASC, a training-free sparse attention method combining value-aware block selection and execution-aware cross-layer memory. Our value-aware block selection integrates pooled query--key relevance with neighboring value contrast, reducing redundancy while preserving query-relevant and distinctive content. Cross-layer memory tracks unserved demand across layers and updates this state according to actual execution, enabling previously underserved blocks to compete under a fixed computation budget. Experiments on 7Scenes and NeuralRGB-D with VGGT and $\pi^3$ demonstrate improved pose estimation and reconstruction quality compared with FasterVGGT, together with up to $2.29\times$ faster inference than dense VGGT. Code is available at https://github.com/kosakayamahoo-design/VASC.
0
cs.CVcs.SDeess.AS Gunwoo Lee, Yoori Oh, Yoseob Han · Oct 1, 2026

This paper tackles video-to-speech (V2S) synthesis, where the central challenge is a one-to-many mapping: visually identical lip movements can correspond to different phonemes. The proposed Watch Your Speech (WYS) framework injects explicit textual conditioning—ground-truth text during training and lip-reader predictions at inference—into a conditional flow matching model via an attention-based fusion module. The key question is whether training-time textual guidance can outperform inference-only classifier guidance while preserving naturalness.

Video-to-speech synthesis aims to generate natural-sounding speech from silent talking-face videos while ensuring phonetic accuracy. A fundamental challenge in this task is the inherent one-to-many mapping problem, where visual dynamics often lack sufficient information to uniquely determine the corresponding utterance. To address this, we propose Watch Your Speech (WYS), a video-to-speech synthesis framework that incorporates textual conditioning as an explicit linguistic cue to mitigate visual ambiguity. Our framework features an attention-based embedding fusion module that synergistically integrates textual context with video sequences, coupled with a conditional flow matching objective for high-fidelity speech generation. Extensive experiments on the LRS2 and LRS3 datasets demonstrate that WYS achieves superior performance, establishing new state-of-the-art results in audio-visual synchronization (LSE-C/D) while maintaining highly competitive textual accuracy (WER). Subjective evaluations further confirm that our model generates speech with near-human naturalness, validating the effectiveness of textual conditioning in content-controlled video-to-speech synthesis. Project page: https://github.com/gunwoo5034/Watch-your-Speech
0
cs.CV Xianda Du, Max Ku, Weiming Ren et al. · Oct 1, 2026

VIEScore2 addresses the shortcoming that most automatic image evaluators return only a scalar quality score without identifying defective regions. It introduces a unified evaluator that represents an image as a sparse $16\times 16$ text grid and jointly predicts perceptual quality, semantic consistency, and defect cell coordinates in a single autoregressive pass. By framing localization as text generation, it enables directly verifiable post-training rewards and supports heterogeneous supervision across generation and editing tasks. The practical value lies in producing human-readable, spatially grounded feedback from one model without task-specific prediction heads.

Existing synthetic image evaluators typically provide only a scalar quality score and do not identify the image regions that support it. We introduce VIEScore2, a unified evaluator for image generation and editing tasks with optional conditioning images. VIEScore2 represents an image as an N x N grid and jointly predicts quality scores and defect locations in a single model pass. Its text-native grid representation provides a common interface for heterogeneous spatial supervision and enables directly verifiable post-training objectives. We train on 38K examples spanning score-only, localization-only, and joint supervision across generation and editing tasks. Starting from supervised fine-tuning, we further apply GRPO to improve defect localization using rewards that combine cell-level Dice overlap, score accuracy, and output-format validity. A parameter-free parser converts the structured predictions into readable explanations. On the primary suite, VIEScore2 achieves an overall-score SRCC of 0.601, compared with 0.491 for Gemini-3-Flash, the strongest zero-shot general-purpose VLM baseline under matched inputs. For defect localization, VIEScore2 outperforms both general-purpose VLMs and specialized spatial evaluators on three of six benchmarks in per-image grid IoU and ranks among the top three on five, including datasets beyond its training sources.
0
cs.LGcs.CVstat.ML Junghwan Park, Woojin Cho · Oct 1, 2026

The paper solves the problem that weight-space task arithmetic is trapped in one architecture because parameter updates share no common coordinate system. Its core idea is the platonic task vector—a model-agnostic object beneath per-model task vectors—and the Universal Task Descriptor (UTD), an architecture-independent matrix of fine-tuning-induced affinity shifts that can be added or negated directly. A closed-form operator folds into the target's last layer, while a low-rank adapter fits the same objective across all layers. The work matters because it enables label-free, weight-free composition of task knowledge across six heterogeneous vision–language and audio–text families.

Models specialized for the same task converge to similar behavior, yet the parameter updates that produce it share no common coordinate system, so weight-space task arithmetic stays confined to a single model and cannot cross architectures without a structural correspondence. Drawing on Plato's allegory of the cave, we hypothesize that these model-specific updates are shadows of one shared, model-agnostic object, which we call the platonic task vector. To make it operational for models that pair an image or audio encoder with a text encoder, we introduce Universal Task Descriptors: matrices whose shape is independent of architecture and embedding dimension, which record a task's functional effect and support addition and negation as matrix operations. Transferring a descriptor into a target means editing the target until it reproduces the descriptor on the task's unlabeled probe images and class-name prompts, requiring no per-image labels. We realize this edit in two ways. First, the descriptor factorizes into a shift field on image embeddings, so a single least-squares solve yields a linear operator that folds into the target's last layer as a weight edit; by linearity, a bank of such operators admits any composition at any strength as a signed sum. Second, a low-rank adapter trained on the same objective reaches every layer and fits compositions jointly, at the cost of one optimization per edit. Heterogeneous models share this object only partially, with a model-specific residual comparable in norm to the shared component, yet cross-model transfer still retains 74-80 percent of the gain of the target's own descriptors. Experiments across six model families, eight classification tasks, and an audio-text setting show that task knowledge transfers and composes across heterogeneous models under both realizations.
0
cs.ROcs.CVcs.LG Chengkai Xu, Yiming Cui, Jiaqi Liu et al. · Oct 1, 2026

End-to-end autonomous driving maps raw sensors directly to control actions, yet its real-world bottleneck lies in the training ecosystem rather than model architecture alone. This survey introduces a Data-Strategy-Platform taxonomy that frames training as an interdependent system: data defines what is learnable, strategies govern how learning aligns with driving objectives, and platforms enable scalability. The authors argue the field must shift from raw data volume to value density, from isolated task-specific learners to universal foundation models, and from static offline training to closed-loop training-testing integration.

Autonomous driving is a cornerstone technology for the future of intelligent transportation, where end-to-end learning has emerged as a transformative paradigm that directly maps multimodal sensory inputs to driving actions through unified differentiable models. While offering advantages, the effectiveness of end-to-end autonomous driving (E2E-AD) is ultimately determined by the quality of its training ecosystem. This paper provides a comprehensive review of training methods and ecosystem for E2E-AD. We introduce a Data-Strategy-Platform taxonomy that conceptualizes training as an interdependent system. The data layer defines what can be learned, the strategy layer governs how learning aligns with driving objectives, and the platform layer supports scalability and continuous evolution. Within this framework, we survey recent advances across data-centric pipelines, learning paradigms, and training infrastructures, and analyze their interplay in shaping model performance, robustness, and deployability. Finally, we reflect on current limitations and articulate a forward-looking vision that emphasizes a shift from data quantity to data value, from isolated optimization to foundation-driven generalization, and from static training to integrated training-testing loops, aiming toward robust, scalable, and trustworthy autonomous driving systems. We maintain a continuously updated repository tracking cutting-edge literature and works at \href{https://github.com/Jiaaqiliu/Awesome-Training-Ecosystem-for-E2E-AD}{Our Project Page}.
0
cs.CVcs.LG Jungmin Lee, Niamat Ullah, Yoseob Han · Oct 1, 2026

EyeTAG tackles temporal jitter in video gaze estimation by making gaze motion an explicit variable rather than an implicit deep feature. It feeds back first-order differences of its own recent predictions as a compact kinematic token, then fuses this with face and eye streams via cross-attention and a causal Transformer decoder. The result is a causal streaming model that cuts mean angular error on Gaze360 by roughly a degree and claims subject-invariant motion via translation-invariant differencing.

Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.
0
cs.LGcs.CV Sidney Bender, Benedikt Kunz, Ahmed Zeid et al. · Oct 1, 2026

Foundation models often rely on spurious correlations, yet generating visual counterfactual explanations is usually slow and entangled. This paper proposes DiDAE, a gradient-free method that wraps a frozen visual foundation model in a conditional diffusion decoder and performs closed-form edits along disentangled dictionary atoms. The approach promises up to 2000× speed-ups over iterative methods while enabling targeted model repair via Counterfactual Knowledge Distillation and unsupervised discovery of shortcut directions.

Foundation models remain vulnerable to spurious correlations and ``Clever Hans'' strategies. Explainable machine learning can find and remove such strategies for classifiers without metadata. For foundation models, no such option exists yet. We propose Disentangled Diffusion Autoencoders (DiDAE). DiDAE wraps a frozen foundation model in a conditional diffusion decoder. A counterfactual is one closed-form edit along a direction of a disentangled dictionary, followed by decoding. The dictionary can be supervised (Procrustes) or unsupervised (Singular Value Decomposition, Sparse Autoencoders). No gradients are needed, so DiDAE is up to 2000 times faster than the state of the art. We evaluate on six datasets, two synthetic and four real-world. In a desiderata-driven benchmark on three of them, its counterfactuals are on par with or better than the state of the art, and they repair downstream classifiers through Counterfactual Knowledge Distillation (CFKD), where they beat metadata-based correction. The same machinery can rank a pretrained dictionary against a trained classifier. It returns the few directions the classifier actually reads, each causally verified by a counterfactual that flips the decision, and repairs the classifier along those a teacher marks spurious. The workflow is plug-and-play in our open-source Peal library we publish alongside the paper. With a public dictionary and a pretrained decoder, all that remains is a cheap linear distillation of the classifier and its own fine-tuning.
0
cs.CV Ozge Mercanoglu Sincan, Anton Pelykh, Edward Fish et al. · Oct 1, 2026

Sign language machine translation remains far behind spoken-language MT because sign languages exploit simultaneous manual and non-manual channels, spatial grammar, and three-dimensional construction. This survey reviews the technical arc from isolated recognition to end-to-end translation and production, embedding every stage in sign language linguistics and deaf-community ethics. The authors argue that progress must be judged not by benchmark scores alone but by whether deaf users actually accept and understand the output, making the manuscript as much a piece of technology assessment as a call for sociotechnical reform.

Sign language machine translation has progressed substantially over the past decade, evolving from isolated sign recognition to end-to-end translation systems. Advances in pose estimation, transformer architectures, and large-scale dataset collection have driven progress, yet challenges remain. Datasets are limited compared to spoken-language resources; evaluation metrics inadequately capture the linguistic quality of output; and models must capture the simultaneous, multi-layered, and three-dimensional structure of sign languages. This manuscript provides a comprehensive review that seeks to balance technical challenges with stakeholder considerations. We examine the linguistic properties that make sign languages computationally unique, trace the evolution of recognition, translation, and production systems, and analyze ongoing technical challenges. Crucially, we address ethical considerations around data governance, community involvement, and appropriate use. Drawing on interdisciplinary perspectives spanning computer vision, sign language linguistics, and deaf studies, our analysis emphasizes that continued progress requires sustained collaboration across these fields and with deaf communities.
0
cs.ROcs.CVeess.IV Pengfei Qi, Haoran Lin, Sizhuang Chen et al. · Oct 1, 2026

UniTrackPLA tackles the problem of unifying instruction-guided navigation and dynamic person tracking—two embodied tasks traditionally handled by separate policies—within a single panoramic vision-language-action framework. Its core idea is to fuse omnidirectional perception with a shared continuous waypoint prediction space, using Panoramic-Aware Encoding to adapt perspective pretrained encoders to 360° observations and World-Action Consistency to adaptively decide when to reuse waypoint chunks or replan. This matters because it removes heading-dependent initialization constraints and offers a step toward general-purpose robots that can follow both linguistic routes and moving humans with one model.

General-purpose embodied robots should support both navigation toward language-specified destinations and dynamic person tracking under arbitrary initial target azimuths. However, existing methods typically rely on forward-facing observations and address these tasks with separate policies, limiting omnidirectional perception and unified closed-loop control. We present UniTrackPLA, a unified panorama-language-action model for instruction-guided navigation and dynamic person tracking. Its Panoramic-Aware Encoding (PAE) preserves the temporal and azimuthal structure of perspective views projected from each panorama, enabling perspective-pretrained visual encoders to process omnidirectional observations. A shared vision-language backbone grounds instructions in the panoramic context and predicts continuous robot-centric waypoint chunks for both tasks. World-Action Consistency (WAC) further predicts action-conditioned future visual states and verifies waypoint prefixes online, allowing reliable actions to be reused while triggering replanning upon inconsistency. We also introduce OmniTrackNav-Bench, comprising 5,000 simulated tracking trajectories, 10,000 simulated VLN routes, and 96 verified real-world routes, providing 919,978 waypoint-supervision instances. UniTrackPLA improves overall tracking SR from 23.50% to 35.00% and Omni-VLN SR/SPL from 13.00%/12.77% to 19.75%/19.29%. Incorporating 76 real-world routes further improves held-out EP@0.2m from 42.92% to 92.08%. Closed-loop experiments on a Go2-W robot demonstrate unified panoramic tracking and navigation across indoor and outdoor environments. The project page is at https://tw5775.github.io/UniTrackPLA.