Muffin - A Shared Memory Buffer for Asynchronous AI Agents
Muffin tackles the overhead and synchronization bottleneck of multi-agent reasoning by replacing full-trace debate with asynchronous sharing of compact intermediate findings through a shared memory buffer $B^{(d)}$. An orchestrator routes queries to either a single agent or a team of parallel agents that read and append selective findings while keeping private reasoning local. The authors claim strong accuracy-token Pareto gains across four reasoning benchmarks and four open-weight language models.
The paper presents a clean architectural idea—selective asynchronous intermediate sharing—and backs it with broad empirical coverage across four benchmarks and four models. Muffin wins 17 of 20 settings and shows large token savings versus debate. However, the evaluation is limited to short-horizon QA and math problems, the comparison against full-trace sharing is conducted only on OlymMATH-Easy, and the asynchronous execution model is mostly conceptual because agents still reason in discrete steps while true latency gains are unmeasured.
The core mechanism is well-motivated: the authors correctly identify that full-message debate overwhelms context windows with irrelevant reasoning while hiding useful partial insights. The ablation in Section 4.3 showing net positive summarizer corrections supports the claim that the final aggregation adds real value beyond simple majority voting. The shared-finding budget experiment also validates that more buffer entries are not always better, lending credibility to the selective-sharing hypothesis.
The headline "asynchronous" claim is technically a red herring: agents still proceed in discrete reasoning steps and the asynchrony is only at the level of dispatching LLM requests, not true event-driven streaming. The paper does not report wall-clock latency or throughput, so the practical benefit of asynchrony is unproven. Additionally, the orchestrator's routing and the agents' decision of what to share are both fully LLM-directed with no learned or constrained mechanism; this introduces uncontrolled variance, leading to larger standard deviations in some settings. The generalization worry is acute because benchmarks use only 100 examples each (except GPQA Diamond at 198), meaning reported margins, especially the 19.2 point leap on OlymMATH-E, could be noisy.
The empirical design is generally solid: same models, same step budgets, and repeated runs help fairness. However, the comparison to Multi-Agent Debate is slightly lopsided because debate is run with full trace sharing while Muffin uses compact findings. The authors do run a full-trace ablation and show Muffin wins on the accuracy-token Pareto front, but this ablation is model- and benchmark-specific. The 77.4% token reduction claim comes from a step-wise analysis on a single benchmark, not a uniform average, which is a bit of cherry-picking despite being labeled as "in the step-wise analysis."
The authors provide an anonymous code repository, detailed prompts in Appendix A, exact model serving instructions (vLLM, H100 counts, tensor parallelism), and report averaged results over three runs with standard deviations. Hyperparameters such as step count ($T=5$), agent count ($N=3$), and retrieval window size are stated. The prompt templates are complete and executable. The main gaps are: (a) no explicit random seed reporting, (b) the orchestrator and agent outputs depend heavily on specific model weights and prompt phrasing that may not transfer cleanly to other API versions, and (c) the anonymous repository link may not persist.
The rise of Agentic AI systems has changed the landscape of artificial intelligence by enabling strong reasoning capabilities through specialized multi-agent interaction with structured self-reflection. However, in target-driven reasoning tasks, single- and multi-agent systems are often on par in terms of performance gains. A major challenge is that failures often arise from an ineffective exchange of critical information during inter-agent interaction and coordination. Different from existing multi-agent debate paradigms that rely on synchronous turn-taking and sharing information across different agents, we propose an inter-agent protocol based on shared Memory bUFFer INference - referred to as MUFFIN - in an asynchronous agentic framework. In the first step, an orchestrator routes queries based on their reasoning demands and delegates demanding ones to a multi-agent setting, where a shared memory buffer provides asynchronous exchange of intermediate findings while an agent's private reasoning remains local. Our experiments show that our framework Muffin outperforms established competitors, achieving the best accuracy in 17 out of 20 of our benchmark settings, and provides a better quality-cost trade-off than multi-agent debate: Muffin improves accuracy by 19.2 accuracy points while using 77.4% fewer tokens, without requiring full-trace sharing.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.