ActICL: Post-Training for Agentic Multimodal In-Context Learning
ActICL attacks the problem of suboptimal multimodal in-context learning by training an MLLM agent to iteratively acquire, inspect, and curate demonstrations through tool-augmented reasoning. Its core idea replaces static similarity retrieval with an agentic loop of search, read, curate, and solve actions, learned via a two-stage post-training pipeline of rejection-sampling SFT and joint reinforcement learning. This matters because it shows that even a modestly sized trained agent can construct contexts that substantially improve its own accuracy and also boost larger frontier answering models.
The paper presents a carefully engineered agentic framework with strong empirical gains and instructive ablations, though its reliance on proprietary frontier models and the conflation of retrieval and answering improvements prevent an unqualified endorsement. Main results are impressive: "ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline." Yet cross-model claims rest on unverifiable systems: "When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks."
The action space and two-stage training protocol are methodically designed and thoroughly validated. The separation of search, read, and curate gives fine-grained control over context construction, and the ablation evidence is compelling: "SFT raises the average score from 0.582 to 0.657, and joint RL alone reaches 0.636." Additionally, the evaluation contexts are not limited to the training domain: "ActICL achieves the best results across all backbones and settings" on held-out interleaved image-text tasks, indicating the training transfers to broader multimodal reasoning.
Several issues temper the central claims. First, while the authors compare against ProMSA, they note that they themselves adapted it rather than running the published system: "We also adapt ProMSA ([Wu et al., 2026b]), a non-ICL multimodal retrieval agent trained with SFT and RL, for ICL context acquisition, using our data and a comparable training cost." This raises questions about whether the baseline is tuned as carefully as ActICL. Second, the comparison to ContextNav conflates backbone capability with retrieval design: "ContextNav uses Gemini3.5-Flash as its policy, while ActICL uses Qwen3.5-9B." A stronger retriever policy may simply reflect a stronger underlying model. Third, joint optimization of acquisition and answering makes it difficult to isolate where gains originate, because the trained model both retrieves and answers; Table 3 partially disentangles this but the headline averages do not.
The evidence is broad but baseline fairness is uneven. The paper covers seven main benchmarks and three backbones with both fixed and adaptive shot counts, and the adaptive regime is genuinely efficient: "Adaptive acquisition commits only 3.93 and 3.26 demonstrations on average, yet it surpasses fixed 4-shot contexts on both models and matches eight shots on Qwen3.5-9B." However, the comparison to the test-time agentic baseline ContextNav is weakened by model mismatch, since ContextNav runs Gemini3.5-Flash while ActICL uses a different backbone. On the positive side, comparisons to LoRA and DARA in Table 2 show ActICL outperforms parameter-efficient fine-tuning on interleaved image-text reasoning, suggesting benefits beyond retrieval alone.
Appendices meticulously describe data construction, resource tiers, and reward design, but the main text does not explicitly commit to releasing code or the 20k constructed training instances. The authors state: "We provide detailed hyperparameter settings and other implementation details in Appendices C–F." Yet reproduction is impeded by reliance on proprietary or hard-to-access frontier models such as GPT-5.6-Luna and GPT-5.6-Sol, and by the substantial compute required for RL on 8B to 30B parameter models. A project page is mentioned — "Project page: https://kaamava.github.io/acticl-project-page/" — but without confirmed artifacts, independent replication would be costly and potentially infeasible.
Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware retrieval methods have emerged to address this problem by allowing incremental retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive retrieval on its own. To this end, we introduce ActICL, which trains an MLLM agent to acquire and use an ICL context via tool-augmented reasoning. An ActICL agent analyzes its own information gaps during iterative reasoning and tool-use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.