Nothing here yet
ECT asks whether inversion-free image editing can be done with just one noise-level evaluation instead of averaging fields at multiple times. It proposes Endpoint-Calibrated Transport (ECT), a training-free method that extracts both an angular semantic tangent and a clean endpoint displacement from a single shared noisy state, then calibrates the update via a closed-form projection. The result is a one-step editor that halves the temporal support of recent one-step baselines and claims competitive quality with lower latency.
Most video pretraining methods recover masked content within a clip rather than predicting the future, and autoregressive token prediction has lagged joint-embedding methods in representation quality. VANE proposes next-frame prediction in a frozen semantic embedding space, using a causal encoder and a lightweight generative decoder to preserve spatial detail, and extends the same objective to still images via local-to-global crops. This unifies image and video pretraining under a single scalable objective centered on temporal evolution, yielding strong gains on anticipation, recognition, and dense benchmarks.
The paper attacks the combinatorial cost of maintaining separate LoRA adapters for every pretrained backbone–task pair. Its core idea, LoRA superposition, is to sum the low-rank updates learned independently on different backbones and apply the same composite update unchanged to every contributing model. The empirical finding is that frozen backbones implicitly select their own component from the sum, enabling a single shared adapter without explicit routing or cross-model alignment. If the phenomenon generalizes, it could enable modular adapter libraries that work across pretrained models without joint retraining.
Federated unsupervised domain adaptation (FUDA) aims to adapt a model to an unlabeled target domain using decentralized labeled sources without centralizing raw data. This paper proposes SKD-ETR, a one-shot FUDA framework that trains a randomly initialized student on the target domain via smoothed knowledge distillation from an ensemble of source teachers, while simultaneously refining those teachers back toward the student through exponential moving average (EMA) updates. The authors claim this bidirectional, communication-efficient loop yields competitive accuracy on standard benchmarks while remaining robust to noisy pseudo-labels.
Multimodal in-context learning remains weak because MLLMs often hallucinate and fail to ground textual outputs in the visual evidence provided in demonstrations.
This work argues that the root bottleneck is poor visual grounding rather than a lack of ICL-specific training data.
The authors propose ProCoRe, which uses proposition-level contrastive reinforcement learning on non-ICL image–caption pairs to sharpen fine-grained visual discrimination.
The result is stronger multimodal ICL without catastrophic forgetting, plus a new personalized-captioning benchmark that forces models to rely on demonstrations instead of lexical priors.
SAM3RL replaces the hand-crafted FIFO memory rule in SAM-based visual trackers with a lightweight controller learned by reinforcement learning. It frames memory-bank eviction as a Markov Decision Process and trains a 3.2 M-parameter MLP with PPO on bounding-box LaSOT sequences to maximize tracking quality. The resulting policy sets a new state of the art on 10 of 12 benchmarks for both SAM 2 and SAM 3 backbones, while remaining fully online and running without extra feature extraction.
Training good latent tokenizers for generative models remains poorly understood, especially how to scale the encoder itself. This paper proposes Relative Prediction Representations (RePR), which supervise patch features to factorize a pointwise mutual information matrix estimated by a conditional diffusion model. The result is a theoretically grounded, negative-sample-free objective that resists feature suppression and claims to be the first to demonstrate compute-optimal tokenizer encoder scaling, matching or beating pixel-space diffusion at equal FLOPs.
Discriminative models generalize to unseen samples, suggesting they capture data-manifold structure beyond their task labels. This paper asks whether that structure can be exposed as a generative signal without retraining, labels, or auxiliary generators. The authors propose the Discriminative Score Function (DSF), which turns the loss-gradient geometry of a fixed discriminative model into an image-space update rule. DSF enables unconditional generation, conditional guidance, inpainting, and zero-shot quantization calibration across classifiers, detectors, and self-supervised encoders.
The paper tackles misalignment between general MRI pretraining objectives and downstream clinical tasks by continually pretraining a 3D prostate MRI ViT encoder. Using LoRA on a prostate segmentation task, the authors evaluate six downstream tasks and use CKA to link representational geometry to performance. The motivation is strong—vision foundation models often need heavy task-specific fine-tuning—but the empirical gains are fractional and inconsistently positive.
Prostate MRI segmentation suffers from regional boundary ambiguity, causing standard global-overlap losses to overlook clinically significant surface deviations. This paper proposes Contrastive Uncertainty Learning (CUL), which treats predictive uncertainty as a representation-level supervisory signal rather than a post-hoc filter: low-uncertainty same-class voxels are clustered in embedding space while high-uncertainty or misclassified voxels are actively repelled. Evaluated on 589 lower-pelvic MR scans across eight anatomical structures, CUL preserves mean Dice while significantly improving surface-sensitive metrics and suppressing extreme errors in hard cases, suggesting practical value for radiotherapy planning and contour-acceptance workflows.
Latent world models based on Joint-Embedding Predictive Architectures (JEPA) offer a scalable route to embodied planning, but existing methods suffer from representational collapse, heavy compute, and weak long-horizon reasoning. This paper proposes MemWM, a lightweight JEPA world model augmented with a content-addressable associative memory bank that supplies high-level subgoals via Adaptive Layer Normalization. By operating at multiple temporal scales within a shared latent space and enforcing slot diversity through complementary losses, the authors aim to boost long-horizon performance without relying on pretrained encoders or doubling model size. Across four control tasks, MemWM achieves an average success rate of 89.9% with only 19M parameters, outperforming heavier baselines such as DINO-WM in aggregate.
Multimodal large language models suffer large accuracy drops when their visual inputs are corrupted by noise, blur, or weather artifacts. This paper asks whether test-time feature alignment—matching activation statistics to clean source moments, a technique that works well for CLIP-like classifiers—can recover that lost performance without labels or weight updates. Across three modern MLLMs and 15 standard corruptions, the authors find it cannot: sample-blind, position-shared corrections recover only about one percentage point of an order-of-magnitude larger drop. The core insight is that corruption shifts in MLLM visual token sequences are nearly orthogonal across images and dominate the residual term in a per-token squared-shift decomposition, whereas pooled classification embeddings exhibit a shared, low-rank shift direction. This matters because it establishes that robustness in generative MLLMs is a per-token restoration problem, not a simple domain-statistics-matching problem.
Steady PDE surrogates on unstructured meshes must resolve multiscale features from a single coarse discretization. This paper proposes routing a mixture of vanilla transformer experts using a learned discrete Hodge-Laplacian controller that measures high-frequency content per vertex. Each expert is biased toward a different spectral scale through auxiliary band targets, and the combined output predicts pressure, velocity, and turbulence fields in one forward pass. On a remeshed AirFRANS benchmark, the architecture cuts mean relative $L_2$ error by roughly a third compared with a Transolver baseline.
Player-1 attacks a core bottleneck in game agents: large-scale demonstrations cover only a narrow slice of dynamic game states, while interactive training in commercial games is hard to scale. The authors extend the computer-use action space with relative pointer motion, sustained keypresses, and composite keyboard–mouse actions. They fine-tune a VLM on just 6,044 curated gameplay trajectories (about 16 hours of control), then run multi-environment PPO in sixteen lightweight synthetic 2D/3D environments with programmatic, verifiable rewards. The central claim is that this synthetic experience yields large Minecraft gains (+27.8 percentage points for the 9B model) and generalizes to unseen games such as ViZDoom and browser-based GameWorld.
Standard Character Error Rate charges every OCR edit equally, whether it is a harmless typo or a silent factual error like changing a name. This paper introduces semCER, which keeps CER's grapheme alignment and denominator but weights each edit by a severity score from an LLM judge, eliminating the need for word segmentation. Validated against human readers in English, Vietnamese, and Chinese, semCER better ranks damaging errors and breaks ties where CER sees identical edit counts, offering a more nuanced tool for OCR and vision-language model evaluation.
3D object composition in simulated environments demands more than static coordinate prediction; complex tasks like hanging, containment, and soft-body settling require iterative interaction and dynamic physical feedback. PhySGA addresses this by embedding LLM/VLM agents directly inside a physics simulator, where an Assembly Agent plans coarse metric adjustments and a Critic Agent refines them via pixel-grounded visual feedback, supported by modular Action Routers and a Self-Evolving Skill Library. The framework matters because it moves beyond layout priors and treats physical construction as a closed-loop agentic process applicable to rigid, articulated, and deformable objects.
Video large language models often hallucinate answers that are not supported by the input footage. Contrastive decoding can reduce this by comparing the original video against a weakened "negative" view, but existing negatives are either predefined or optimized for query reconstruction rather than the model's actual visual reliance during answer generation. ARP-CD introduces a training-free fix: it treats the attention mass on visual tokens at the answer-prediction position as a differentiable optimization target, then uses bounded gradient ascent to perturb the input visual hidden states and create a negative branch for contrastive decoding. The authors evaluate the idea across four Video LLMs and four benchmarks, supplementing headline results with ablations and a mechanistic analysis of first-token logit margins.
This paper identifies a gap between standard visual grounding—which assumes enough dialogue context to pinpoint a single object—and real-world interactions where multiple referents remain plausible. It proposes Candidate Set Visual Grounding (CSVG) and introduces the CandVG benchmark to evaluate how the set of plausible referents evolves across dialogue turns. Evaluating 18 vision-language models shows that stronger single-object grounding does not translate to better set estimation, with models often failing to generate full candidate sets or committing prematurely to one candidate.
Multimodal code intelligence demands programs that are correct, well-structured, and visually faithful, yet existing training signals are either too coarse or confined to single tasks. This paper introduces PrismaForge, which synthesizes per-instruction evaluation rubrics that separately assess code-level and visual-level criteria, then uses them to filter training data and reward reinforcement learning. The resulting dataset, PrismaCoder-2M, and the trained 9B and 35B models advance the open state of the art and compete with proprietary systems on charts, webpages, and scientific graphics.
Document understanding has been evaluated through isolated silos—layout detection, OCR, reading order, and image captioning—making it impossible to audit whether a single coherent page parse exists. This paper proposes Read-Parse-Describe, a unified benchmark built from 2,402 ACM scientific documents that forces models to jointly output bounding boxes, semantic labels, reading order, recognized text, caption links, and alternative-text descriptions for visual elements. To combat spatial drift in compressed vision-language tokens, the authors introduce SARA, a lightweight residual adapter that re-injects crop-aligned multi-layer visual features into frozen VLMs.