Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
398 papers in cs.CV
Trending mixes fresh papers with community signal.
0
cs.CVcs.AIcs.LG Taha Sabir, Alberto Montero Molina, Thinh Quang Ly et al. · Oct 14, 2026

Medical image parsing requires a single model to produce heterogeneous outputs—categorical labels, bounding boxes, counts, numeric estimates, and free-text reports—across disparate imaging modalities. This work studies task-specific adaptation of a shared 4-bit Qwen3.5-9B backbone using lightweight LoRA adapters, where a question-only GLiClass router selects the adapter, image budget, and prompt for each case on the FLARE 2026 Task 3 benchmark. The core finding is that adaptation recipes should vary by task—rank, resolution, and prompts allocated according to demand—rather than applied uniformly.

Adapting a single vision-language model to medical image parsing requires training choices that reflect the demands of each task. We study task-specific adaptation with Qwen3.5-9B for FLARE 2026 Task 3, covering six task families and seven imaging modalities. Our system couples a shared, frozen 4-bit backbone with LoRA adapters selected from the input question. We customize adapter ranks and prompts, rebalance rare classification targets, and supplement regression training with dermatology classification examples. On the fixed internal holdout, the task-specific adapters achieve a dataset-macro balanced accuracy of 0.7791 for classification, a dataset-macro micro-F1 of 0.6286 for multi-label classification, a counting mean absolute error (MAE) of 11.73, a dataset-macro detection F1 of 0.8914, a dataset-macro regression MAE of 9.78, and a report-generation CRIMSON score of 0.8204. On the organizer's 1,703-example hidden-validation set, the submitted system achieves 0.8496 classification balanced accuracy, 0.5338 sample-averaged multi-label F1, and 14.5950 regression MAE. The final 30.1 GB container averages 3.1 seconds per case, including model loading, and produces identical outputs across repeated verification runs. The benefits vary across tasks, supporting the use of task-specific adapter configurations and image budgets within a shared medical vision-language model. Code is available at https://github.com/TahaSabir0/FLARE26-Hexapus.
0
cs.CVcs.AI Siqi Yang, Yimu Wang, Ziqi Cai et al. · Oct 14, 2026

PAWA tackles multi-agent world modeling in shared environments like Minecraft, where one player's actions change what every other player sees—not just in their local field of view, but in directions they are not currently looking. The core idea is to jointly forecast each player's point-of-view (POV) video alongside a world-stabilized panoramic video, conditioned on all players' actions. The model uses a dual-tower architecture with ray-aligned cross-attention that links tokens viewing the same world direction regardless of whether they appear in rectilinear POV or equirectangular panorama coordinates, plus a quadratic-phase player encoding that scales identities as $m^2$ instead of $m+1$. The work also introduces a scalable dataset pipeline that records gameplay without graphics clients and renders replays offline, supporting more than 64 simultaneous players.

A multiplayer world model must predict how the independent actions of several players change a shared environment, including changes outside a player's current field of view. A point-of-view (POV) video captures such changes only after they enter its field of view, whereas panoramic video records the surroundings in every viewing direction and has served as an intermediate representation of the world state in recent world models. Learning this task therefore requires synchronized POV and panoramic videos of interacting players together with their actions, yet no existing dataset provides this combination in Minecraft, an established testbed in which players act independently in a shared, modifiable 3D world. We present PAWA, a dataset and framework for action-conditioned panoramic multiplayer world modeling. The dataset pairs each player's POV video with world-stabilized panoramic video and aligned actions. Its distributed pipeline decouples CPU gameplay capture from GPU rendering, scales beyond 64 simultaneous players, and re-renders each recorded session under new observation configurations without replaying the gameplay. On this dataset, we develop a dual-tower model that jointly predicts POV and panoramic video and couples the two streams through cross-attention that aligns tokens observing the same direction in a shared world frame. We further introduce a quadratic-phase player encoding that provides many more player codes than existing rotary codes at the same embedding dimension. Experiments show that PAWA outperforms state-of-the-art multiplayer world models in POV prediction quality while also producing high-fidelity panoramic predictions. The dataset and collection pipeline are open source to support further research on multiplayer world models.
0
cs.CVcs.AI Imen Bajar, H\'eber Hwang Arcolezi, Antoine Boutet et al. · Oct 14, 2026

This paper asks whether releasing learned embeddings from pretrained brain MRI encoders—instead of raw scans—actually protects patient privacy. The authors audit five diverse encoders on two longitudinal datasets, measuring patient linkage, re-identification, and sex-attribute inference under an explicit attacker model. They find that several embeddings retain enough subject-specific structure to enable rank-1 re-identification above 90% on both datasets, and that sex can be inferred with balanced accuracy up to 95.5%. The core takeaway is that embedding release alone is insufficient to establish anonymity.

Pretrained representations are increasingly used to share and reuse information from medical images without exposing the original scans, but it is unclear how much patient-specific information these representations retain. We investigate this question for brain magnetic resonance imaging (MRI) by evaluating five frozen pretrained encoders spanning different architectures, pretraining objectives, and training domains on two longitudinal T1-weighted datasets, OASIS-2 and MIRIAD. Under an attacker model with access to released embeddings, the corresponding public encoder, and auxiliary MRI scans, we evaluate patient linkage and sensitive-attribute inference. Across the evaluated representations, we observe substantial subject-specific information: rank-1 patient linkage reaches up to 99\% on MIRIAD and exceeds 90\% on OASIS-2 for the highest-risk representation, despite the absence of an identification objective during encoder training. We further show that the released embeddings retain information about a demographic attribute, with recorded-sex prediction reaching balanced accuracy of 90.9\% on OASIS-2 and 95.5\% on MIRIAD. Overall, our results demonstrate that replacing raw brain MRI with learned embeddings does not, by itself, eliminate patient-linkage or attribute-inference risk, and is therefore insufficient on its own to establish anonymity.
0
cs.CVcs.AIcs.LG Lorenzo Venturoli, Nedko Savov, Xi Wang et al. · Oct 14, 2026

StateCraft tackles the long-horizon inconsistency problem in video world models by explicitly decoupling a persistent, structured abstract state from the visual rendering pipeline. A finetuned vision-language model maintains and updates a graph-structured abstract state in response to actions, including off-screen changes, while a video diffusion model generates each frame conditioned on that state. This enables consistent revisits, direct state editing, and large empirical gains on a new GUI-world-model benchmark measuring memory and controllability.

Video world models aim to simulate complex environments with realistic appearance and dynamics. Maintaining consistency over long interactions requires tracking past actions and understanding how those actions affect the environment. Video world models can only generate partial observations of the entire world at a time. Remembering past observations alone may not be sufficient in every scenario, as they may no longer reflect the world after a sequence of actions. We introduce a paradigm that decouples a persistent, updatable abstract state from the visual state. The abstract state keeps track of the entities' changes as the world evolves, and is used to condition the visual state, which represents the current view. We propose StateCraft, a novel architecture that pairs a language model with a video diffusion model to evolve the dynamic world. We evaluate our approach on generative interfaces, which pose a substantial challenge because a single action may change parts of the state that are not currently visible. To support training and evaluation, we develop a data collection pipeline that produces informative, densely annotated trajectories at scale without human or agent supervision, together with a benchmark measuring generation quality, memory, and controllability. On this benchmark, our approach improves surpasses the state-of-the-art baselines. We demonstrate that StateCraft maintains high self-consistency and strong generation quality over long trajectories. Because the abstract state is explicit, it can be edited directly, and generation follows edits to states never observed in the data. Finally, a proof-of-concept 3D experiment, in which actions change the illumination of an unobserved room, shows that the paradigm can also be applied to spatial environments. Project page: https://insait-institute.github.io/StateCraft/.
0
cs.CVcs.AI Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang et al. · Oct 14, 2026

LiFi targets the task of generating metric LiDAR point clouds from cheap onboard multi-view camera images for autonomous driving. The core idea is a latent diffusion model guided by two decoupled branches: a geometry branch that aligns dense monocular depth estimates to the target LiDAR scanning pattern via a reverse sampling algorithm and uncertainty-modulated encoding, and a semantic branch that aggregates global cross-view context. If the approach holds up, it could enable large-scale data augmentation, cross-modal simulation, and sensor completion without requiring expensive LiDAR capture at deployment time.

Generating LiDAR point clouds from multi-view camera images is a valuable task with applications in controllable data synthesis, cross-modal simulation and various perception tasks. However, most existing methods are not specifically designed for this task, and the quality of their generated LiDAR remains limited. Moreover, these methods typically focus only on semantic cues in images or generate LiDAR from a monocular camera image. In this work, we introduce LiFi (LiDAR From images), a dedicated framework for generating visually and geometrically aligned LiDAR scenes from multi-view camera images. LiFi guides the latent diffusion process through two complementary branches: geometry and semantics. The geometry branch constructs scene features from images through a reverse sampling algorithm and an uncertainty-aware geometric encoder combined with DepthAnything3, whereas the semantic branch provides high-level semantic representations via a cross-view interaction module. We further introduce a dual-branch balanced classifier-free guidance (CFG) strategy, which enhances the model's conditional generation capability while preserving the independent learning and collaborative controllability of the two branches. Extensive experiments demonstrate that LiFi outperforms state-of-the-art methods in generating high-fidelity LiDAR scenes. The code will be publicly available at: https://github.com/szzhou88/LiFi.
0
cs.CVcs.AI Shuofeng Sun, Wenzhao Zheng, Yunpeng Zhang et al. · Oct 14, 2026

SceneAE proposes a sparse 3D-anchored latent representation for scene reconstruction and camera-controllable generation. Instead of viewpoint-aligned video latents, each latent couples an explicit 3D point with an implicit feature, enabling queryable rendering from arbitrary cameras. A two-stage diffusion model decouples geometry generation from content synthesis, improving camera control and cutting computational cost roughly in half compared with dense-grid baselines.

Recent video world models have increasingly adopted 3D memory mechanisms to integrate historical observations into a lightweight and persistent scene representation. This naturally raises a complementary question: if 3D representations can serve as an effective mechanism for memorizing observed scenes, can they also be directly used as the latent space for reconstructing and generating scenes? Motivated by this idea, we represent a scene as a compact and unstructured 3D-anchored latent space (\textbf{SceneAE}). Each latent combines an explicit 3D position with an implicit content feature and is spatially grounded to describe its local scene region, forming a sparse latent point cloud. Unlike conventional video-VAE latents that describe how a scene appears from particular viewpoints, our representation directly describes the scene itself, making it naturally applicable to both reconstruction and generation. For reconstruction, once the scene has been generated, our persistent 3D representation can be projected and decoded into RGB-D observations by different target cameras, without the need to regenerate a video for each new camera trajectory. For generation, we leverage the geometry-content separation to first generate camera-controlled 3D geometry and then synthesize content conditioned on it, reducing camera-appearance entanglement and improving camera controllability. Experiments show that this 3D latent achieves superior generation quality, computational cost, and camera controllability.
0
cs.CV Chih-Chung Hsu · Oct 14, 2026

RGB vision-language-action models struggle to distinguish waste objects that appear identical in color but differ spectrally. SPECTRA-VLA learns a compact, base-orthogonal spectral residual that enriches frozen visual tokens at their native width without retraining the visual encoder or action decoder. On the SpectralWaste dataset, the method improves category balanced accuracy across 86 held-out acquisition sessions while retaining within-category spectral shape better than category-level codes.

Hyperspectral imaging distinguishes RGB-similar objects, but a pretrained vision-language-action model needs a state its frozen decoder can use. SPECTRA-VLA provides a native-width spectral upgrade without retraining the visual encoder or action decoder. A learned continuous residual enriches each target token through its own normalized false-color reference; projection, norm control and validation-calibrated injection regulate the change. Across 86 held-out acquisitions, category balanced accuracy rises from 0.812 to 0.883. With identical residuals and scale, own-token addition improves reuse of an external category-derived probe on frozen decoder logits: 0.701 versus 0.590 with pooled replacement, at similar image-disjoint recognition. Five grouped tests covering the same 86 acquisitions recover within-category spectral shape with $R^2=0.802/0.337$ from continuous and posterior-code states. Category-matched spectral swaps further show that own-token composition approximately doubles the target-specific recovery increment after frozen decoding. Spectral enrichment and receiving-token preservation thus work together at the pretrained interface.
0
cs.CV Tianyu Zhang, Haiyi Ren, Zhilun Zhang et al. · Oct 14, 2026

Arctic coastal polynyas drive critical ocean-atmosphere heat exchange and sea-ice production, yet mapping them from passive microwave sea ice concentration (SIC) is fraught with ambiguity because narrow coastal openings share SIC ranges with the marginal ice zone and open ocean. The authors develop a daily mapping framework that couples hierarchical SIC partitioning, connected-component analysis, and differentiated region growing to separate enclosed low-concentration polynyas from ocean-connected water, then apply it to September–May seasons from 2002/03 to 2024/25. Their central finding is that absolute polynya extent is highly sensitive to both the upper SIC threshold and the input product—varying by up to roughly 65% across thresholds and showing 68% MAPE across products—while broad temporal patterns remain comparatively stable. This makes a clear case that reproducible long-term monitoring requires a fixed, explicitly documented configuration rather than ad hoc threshold choices.

Accurately delineating Arctic coastal polynyas from sea ice concentration data remains challenging because of complex coastal geometry and spatially heterogeneous ice conditions. We develop a daily sea ice concentration (SIC) mapping framework based on hierarchical partitioning, connected-component analysis, and differentiated region growing for polynya and open-water regions. Applied to September-May ice seasons from 2002/03 to 2024/25, the framework achieves a mean overall accuracy of 96.74% and a mean kappa coefficient of 0.93 in the 63-scene visual assessment. Sensitivity experiments reveal a consistent pattern: the absolute magnitude of SIC-defined polynya extent is strongly configuration-dependent, whereas major temporal features are comparatively more stable. Relative to a 75% upper SIC threshold, seasonal extent integrals differ by 13.48%, 32.47%, and 65.04% at thresholds of 80%, 85%, and 90%, respectively, while the broad interannual structure remains recognizable. The response is also strongly regional and seasonal, with the largest threshold sensitivity in the Central Arctic Ocean, very weak sensitivity in the Gulf of St. Lawrence, and greater pan-Arctic curve separation from January onward. At an 80% threshold, product-comparison MAPE is 68.19% between the 6.25 and 25 km products and 11.47% between the 12.5 and 25 km products under their respective comparison periods and reference definitions. Regional product disagreement is likewise heterogeneous. Together, these results show that reproducible long-term polynya monitoring requires a fixed and explicitly documented combination of SIC product, threshold, and spatial classification rules, while broad temporal patterns can remain informative even when absolute extent differs substantially.
0
cs.CV Rui Chen, Xingyu Chen, Pengxin Xu et al. · Oct 14, 2026

HyrCap addresses inaccurate temporal localization and low-quality proposals in long untrimmed videos by treating dense query-based predictions as an over-complete temporal event hypothesis space rather than final detections. It hybridizes complementary video representations—InternVideo2 for strong proposal quality and V-JEPA2 for localization reliability—and recalibrates proposal confidence via cross-representation consensus, localization quality estimation, and redundancy suppression. This matters because reliable temporal proposals are framed as a critical interface between detection and LLM-driven semantic understanding, motivating a new two-stage evaluation protocol that decouples localization from segment-level semantics.

Temporal action detection in long untrimmed videos suffers from inaccurate localization and low-quality proposals, especially with sparse annotations and ambiguous boundaries. For open-ended video understanding and LLM-driven event analysis, reliable proposals connect temporal localization, event annotation, and semantic understanding. However, conventional temporal action detection (TAD) pipelines lack explicit modeling of proposal localization quality, cross-representation support, and inter-proposal relations. We propose Hybrid Rank-Calibration of Action Proposals (HyrCap), a proposal-centric query-based TAD framework for temporal event understanding. Hybrid combines complementary video features with different localization cues, while Rank-Calibration adjusts proposal ranking confidence to reflect localization reliability and candidate competition. HyrCap treats dense query-based predictions as an over-complete temporal event hypothesis space. Cross-Representation Consensus Calibration, Localization Quality Estimation, and Proposal Redundancy Suppression improve proposal ranking reliability, localization quality, and candidate discrimination. Inspired by LLMs' capabilities in fine-grained semantic recognition, event description, and reasoning, we introduce HyrCap for Semantic Action Understanding, using calibrated proposals for standard TAD and as inputs to LLM-based semantic understanding. We define a staged evaluation protocol to separately measure event localization, segment-level semantic understanding, and combined event understanding performance. Experiments on public benchmarks show that HyrCap achieves state-of-the-art performance across multiple metrics and provides a finer-grained paradigm and evaluation protocol for query-based event segmentation and annotation.
0
cs.CVcs.LG Zeyang Zhang, Linxiang Gao, Meiqi Gu et al. · Oct 14, 2026

The paper tackles whether iterative image generation with flow matching can actually perform deep sequential reasoning, or whether the many sampling steps are merely an illusion of compute. It proves that vanilla flow matching collapses to constant-depth circuits because each ODE step changes state by only $O(1/K)$, but shows that self-conditioning—feeding back an endpoint prediction as an internal scratchpad—breaks this barrier and can reach $\mathsf{P/poly}$ and even $\mathsf{PSPACE/poly}$. The result is a sharp theoretical and empirical argument that how information flows between sampling steps matters more than how many steps you take.

Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading. We prove that as long as the model outputs satisfy certain mild regularity conditions, the trajectory of the vanilla flow-matching sampler can be simulated by a constant number of sequential rounds of parallel model evaluations. Consequently, additional sampling steps cannot increase the expressive power of a constant-depth backbone model under the standard complexity-theoretic model of transformers. We then reveal that self-conditioning, which conditions the model on the previous step's estimate of the final image, plays a previously unrecognized role in removing this bottleneck. Controlled experiments on diverse visual-reasoning tasks show that flow matching with self-conditioning achieves higher accuracy and greater reasoning depth than vanilla flow matching, and its performance improves with more sampling steps. Theoretically, we prove that with self-conditioning, a constant-depth backbone can achieve expressive power equivalent to that of space-bounded Turing machines. Our study suggests a design principle for future visual reasoning research: carry more information between sampling steps, so that additional steps translate into deeper reasoning.
0
cs.CVcs.GR Chia-Ming Lee, Yu-Jou Hsiao, Ming-Ching Chang et al. · Oct 14, 2026

Cameras and foundation models recognize what a 3D scene contains, but not what it is made of: surfaces of the same colour can be different materials. Spectral World Renderer tackles this by unifying world generation and hyperspectral unmixing inside 3D Gaussian splatting, so that each Gaussian carries material abundances decoded before ray compositing. The resulting SWR-Bench provides eight synthetic worlds with exact per-pixel material labels, enabling verifiable decomposition benchmarks and real-time material queries.

Cameras and the foundation models built on them recognize what a 3D scene contains, but not what it is made of: surfaces of the same colour can be different materials. Hyperspectral unmixing explains each spectrum as a mixture of material signatures, yet in 3D it is applied to rendered images after the views are composited, or tied to slow implicit fields, and no multi-view data with known composition exist to check the result. We present the Spectral World Renderer (SWR), which unifies world generation and unmixing. A coding agent builds executable 3D worlds that are rendered with measured spectra, so every hyperspectral image comes with exact labels and sub-pixel proportions; the resulting SWR-Bench provides eight indoor and outdoor worlds in which decompositions can be verified. SWR then unmixes inside rendering: each Gaussian of a compact 3D representation is decoded into material abundances before compositing, so one render yields spectra and material maps, and any surface can be queried by material name, a clicked pixel or a reference spectrum. Without material labels, SWR identifies materials best in all eight worlds, has the best mean PSNR on real captures, trains 3.7-12.9x faster than unmixing baselines, renders 34.9 labelled views/s and separates same-coloured materials.
0
cs.CV Yuanyuan Gao, Zhirui Liu, Yuning Gong et al. · Oct 14, 2026

Holi-Spatial 360 targets a key data bottleneck for spatial intelligence: creating large-scale, photorealistic, semantically grounded 3D indoor environments without costly LiDAR or dense video capture. The pipeline converts sparse real-estate panoramas into coherent 3D Gaussian Splatting scenes by anchoring optimization with cross-view correspondences from a large reconstruction model, then densifies trajectories with adapted image and video generative priors. The output is Holi360, a dataset of over 1,000 multi-room scenes with dense RGB-D, 3D instance annotations, and 2 million spatial question-answer pairs, plus a benchmark for VLM spatial reasoning.

Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial memory. Adapted single-image and video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360 benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial reasoning capabilities.
0
cs.CV Dikai Li, Weijia Wang, Yuan-Gen Wang · Oct 14, 2026

Active visual recognition fixes both field of view and observation count, yet images vary in the scale and amount of evidence they need. This paper extends the public AdaptiveNN-DeiT-S pipeline with a controllable observation interface that jointly adapts fixation geometry via a learned scale action, local evidence quality via scale-gated foveation and a residual detail-token branch, and observation count via label-grounded confidence-conditioned stop heads. The resulting single checkpoint exposes threshold-controlled accuracy-compute trade-offs without retraining, directly addressing the mismatch between uniform observation budgets and heterogeneous image complexity.

Visual recognition models usually process a fixed view under a fixed computation budget, although images often benefit from different scales and amounts of spatial evidence. Active vision can adapt by observing a coarse scene and then selecting informative fixations, but the public AdaptiveNN-DeiT-S pipeline still fixes both field of view and observation count. We extend this fixed-step recognizer with a controllable observation interface that adapts both the field of view and the number of fixations to the input. Specifically, the model predicts a per-fixation scale that sets its field of view, refines each selected crop with optional scale-gated foveation and a residual high-resolution detail-token branch, and uses label-grounded stop heads for input-dependent early termination. Experiments across fine-grained benchmarks, full ImageNet-1K, and ImageNet-A/R-derived classification splits show that the interface remains competitive with fixed-step references while exposing a controllable observation budget. The same checkpoint enables threshold-controlled accuracy-compute trade-offs without per-threshold retraining. On sequential ImageNet-1K, the learned stop rule outperforms fixed-step truncation of the same checkpoint, sustaining 81.9% top-1 using 2.45 of the four focus steps and exceeding three-step truncation at lower cost. On each benchmark, a variant combining scale control and detail-token fusion achieves the highest accuracy among the tested component-added non-exit variants. Our code is available at https://github.com/GZHU-DVL/LI-Dikai.
0
cs.CV Jia Chen, Ruyang Zhao, Jun Bai et al. · Oct 14, 2026

Single-image reconstruction of standalone garments remains under-supported by existing benchmarks, which mostly target clothed humans or rigid objects. MagicGarment introduces a dataset of 1,518 synthetic assets across 17 composite types, paired with aligned multimodal annotations and a unified evaluation protocol. The authors also propose a garment-aware baseline that adapts a frozen Hunyuan3D backbone via input-level semantic refinement (GSR) and a dynamic timestep- and routing-conditioned LoRA (FTR-LoRA), demonstrating that domain-specific adaptation improves over generic 3D priors on this benchmark.

Single-image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin-shell geometry, open boundaries, and layered structures. Existing datasets and evaluation protocols provide limited support for systematic evaluation in this setting. We introduce MagicGarment, a benchmark for single-image standalone 3D garment reconstruction with 1,518 professionally constructed assets across 17 composite garment types, aligned multimodal annotations, and a unified evaluation protocol. We further establish a garment-aware reference baseline by adapting a pretrained image-to-3D backbone at two levels: Grounded Semantic Refinement (GSR) provides garment-focused visual conditioning, while Flow-Time-Route LoRA (FTR-LoRA) adapts the frozen 3D generative backbone using flow-timestep, token-routing, and expert-load information. Experiments demonstrate MagicGarment's benchmark utility and the effectiveness of garment-aware adaptation. Project page: https://1019538157.github.io/MagicGarment/.
0
cs.ROcs.CV Yunze Liu, Jing Wang, Shenrui Wu et al. · Oct 14, 2026

Robota-Embed tackles the problem of retrieving relevant physical experience for robots by learning a single embedding space that jointly encodes vision, language, and action from both the current state and a future trajectory segment. The core insight is that a useful retrieval key must capture not just task intent and visual scene, but also the robot's ongoing motion and execution stage, because the goal is to retrieve an appropriate continuation rather than simply a similar observation. A new multi-source benchmark and two downstream applications—in-context policy conditioning and memory evolution around a frozen large policy—show that this unified, action-aware representation significantly outperforms vision-language-only baselines and improves both success rates and token efficiency.

Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware retrieval space can support robot in-context learning and task-local context evolution.
0
cs.CV Yiliu Xu, Xinjin Li, Yu Ma · Oct 14, 2026

End-to-end accuracy on multi-image vision-language tasks hides whether failures stem from visual retrieval, schema understanding, or downstream composition. This paper introduces a within-item $2 \times 2$ factorial Oracle design in which a model performs a 16-image schema-application task under four matched conditions, receiving verified textual schema (Oracle-A), verified ordered target appearances (Oracle-B), both, or neither. The results localize the dominant recoverable deficit to upstream ordered instance recovery—Oracle-B rescues strict accuracy by 54.6 percentage points—while supplying the schema alone barely helps, suggesting that current VLMs often lose the relevant visual facts before any codebook reasoning begins.

Modern vision-language models can accept multi-image contexts, but aggregate accuracy rarely reveals where a multi-image reasoning pipeline fails. We study a controlled schema-application task in which a model must locate two target characters in a 16-image sequence, identify their visual attributes, consult an item-specific visual codebook, and compose two ordered nonce code names. We introduce a within-item 2-by-2 factorial Oracle design: Oracle-A supplies a verified textual schema, Oracle-B supplies a verified ordered target-appearance representation, and both interventions retain the original images and final question. Across 72 new items and 1,188 analyzed model responses, Oracle-B increased strict exact accuracy by 54.6 percentage points (95% confidence interval: 46.3 to 63.0) and outperformed Oracle-A by 60.6 points (Holm-adjusted p-value no greater than .0002). An auxiliary retrieval diagnostic reached only 49.1% ordered-appearance accuracy even after removing code-name generation. Providing both Oracle sources yielded 100% accuracy across all models. These results localize the dominant recoverable deficit to upstream ordered instance recovery rather than primarily to downstream schema use and composition.
0
cs.CV Haoxiang Rao, Fang Zhao, Zhao Wang et al. · Oct 14, 2026

Industrial anomaly detection is bottlenecked by scarce anomalous samples, which limits supervised training and defect understanding. This paper proposes CORE, a training-free framework that leverages Large Multimodal Models to synthesize realistic anomalies in both zero-shot and one-shot regimes through structured prompt optimization. The key idea is to decompose prompts into BASE, VARIANT, and NEGATIVE components and refine them via a self-supervised contrast-and-reflect mechanism without human annotations or task-specific fine-tuning. If effective, this could remove the need for costly real defect collections and enable open-world adaptation.

Industrial anomaly detection (AD) is bottlenecked by the scarcity of anomalous samples, making high-fidelity anomaly synthesis crucial for improving downstream performance. However, existing methods either rely on scarce real anomaly references or auxiliary data, while lacking complex reasoning and iterative optimization capabilities for generating realistic defects in open-set scenarios. To address these challenges, we propose CORE, a cost-efficient and training-free prompt optimization framework based on Large Multimodal Models (LMMs), which unifies zero-shot and one-shot anomaly synthesis. CORE first introduces an Anomaly Context Grounding module to transform sparse user context into defect-aware generation context, which is subsequently instantiated as structured anomaly prompts that disentangle normal-region preservation from anomaly-specific variation, enabling controllable and faithful defect generation. Furthermore, CORE autonomously refines prompts through a self-supervised contrast-and-reflect mechanism. This design empowers general-purpose LMMs to synthesize realistic anomalies under both zero-shot and one-shot settings without task-specific training. Extensive experiments on MVTec-AD and VisA demonstrate that CORE synthesizes realistic and diverse anomalies, yielding superior improvements in downstream AD performance. Furthermore, its model-agnostic and plug-and-play paradigm enables effortless integration with diverse LMMs, facilitating adaptation to unseen open-set scenarios.
0
cs.CVcs.AI Liang Yu, Tao Zhu, Xiangqing Lu et al. · Oct 14, 2026

Real-time monocular human pose estimation breaks down when the lower body is occluded or the subject steps out of frame. This paper introduces a part-decoupled temporal fusion framework that augments monocular video with sparse plantar-pressure and foot-mounted acceleration from smart insoles. The core insight is that cheap, wearable foot sensors provide complementary physical constraints exactly where monocular vision is weakest, enabling robust pose and gait recovery under severe visual degradation.

Real-time monocular human pose estimation is an important sensing component for human-centered intelligent systems. While recent camera-based methods have achieved impressive accuracy, fundamental challenges persist, particularly when visual observations are incomplete, such as during lower-body occlusion or short out-of-frame intervals. In these challenging but common scenarios, vision-only systems often struggle to reconstruct physically plausible lower-limb motion and reliable foot-ground contact. Due to the highly unobtrusive and wearable nature of smart insoles, this paper investigates whether simple plantar pressure and acceleration sensors can effectively resolve these visually induced limitations. To systematically investigate this, we first collect a synchronized multimodal dataset and then propose a part-decoupled temporal fusion framework. The proposed model fuses foot-sensing features into the lower-body branch to compensate for incomplete visual evidence, while a lightweight coordinator exchanges information between the upper- and lower-body temporal representations to maintain whole-body consistency. The framework is evaluated with three representative real-time visual backbones based on 2D keypoints, 3D landmarks, and SMPL-based human mesh recovery. We evaluate our framework on three representative real-time visual backbones. Quantitative experiments demonstrate that incorporating simple foot sensors consistently improves pose accuracy, motion smoothness, and gait-related measurements over vision-only baselines, with the most significant gains observed under occlusion and short visual-dropout conditions. Ultimately, our results demonstrate that unobtrusive foot sensing provides vital and complementary physical observability when camera observations are temporarily unavailable.
0
cs.CV Ke Li, Tianyi Wang, Enning You et al. · Oct 14, 2026

LED-BENCH targets a realistic but under-evaluated setting: answering engineering questions from complete, large-format drawings in mechanical, architectural, and transportation domains. The benchmark contains 107 sheets and 377 expert-verified questions that require models to localize fine-grained evidence and integrate information across distant regions or views. Its central finding is that current multimodal models remain far from solving this problem—the best evaluated model achieves only 52.65% accuracy, and performance consistently degrades on harder reasoning tasks.

Real-world engineering drawings are often large-format and information-dense. Answering practical engineering questions therefore requires both fine-grained understanding of engineering information and complex reasoning over objects, relations, and evidence distributed across different regions or views. Existing benchmarks cover a range of engineering-drawing understanding tasks involving local regions, individual views, specific elements, or more constrained visual settings. However, systematic evaluation of understanding over realistic, large-format engineering drawings remains limited. To address this gap, we introduce LED-BENCH, a multidisciplinary benchmark designed to evaluate fine-grained understanding and complex reasoning over large-format engineering drawings, spanning the mechanical, architectural, and transportation domains. Extensive evaluation of 11 representative multimodal large language models, including 8 proprietary models and 3 open-source models, reveals substantial remaining difficulty: GPT-5.6 Sol achieves 52.65% average accuracy, performance varies across engineering domains, and all evaluated models perform worse on questions requiring broader evidence integration. These results highlight fine-grained understanding, evidence integration, and complex reasoning over large-format engineering drawings as persistent challenges for current multimodal models.
0
cs.CVcs.CL Yanshu Li, Ruixiang Tang, Junyi Jessy Li et al. · Oct 14, 2026

ActICL attacks the problem of suboptimal multimodal in-context learning by training an MLLM agent to iteratively acquire, inspect, and curate demonstrations through tool-augmented reasoning. Its core idea replaces static similarity retrieval with an agentic loop of search, read, curate, and solve actions, learned via a two-stage post-training pipeline of rejection-sampling SFT and joint reinforcement learning. This matters because it shows that even a modestly sized trained agent can construct contexts that substantially improve its own accuracy and also boost larger frontier answering models.

Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware retrieval methods have emerged to address this problem by allowing incremental retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive retrieval on its own. To this end, we introduce ActICL, which trains an MLLM agent to acquire and use an ICL context via tool-augmented reasoning. An ActICL agent analyzes its own information gaps during iterative reasoning and tool-use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.