MemFit: Efficient Long-Term Agentic Memory
MemFit tackles the high cost of LLM-driven memory consolidation by keeping the LLM out of the write path entirely. It stores each conversational turn verbatim in an append-only store, indexes via BM25 and dense embeddings, and retrieves through a multi-stage pipeline combining lexical matching, dense similarity, pseudo-relevance feedback, and cross-encoder reranking. The work claims state-of-the-art accuracy on three long-term conversational benchmarks while cutting memory construction cost and latency several-fold.
MemFit presents a conceptually elegant architecture whose efficiency claims are well-supported, but its narrative of across-the-board accuracy dominance overstates the evidence. The append-only storage design is a genuine advance over rewrite-and-compress pipelines, and the multi-path retrieval pipeline is carefully engineered. Yet the paper's own results show that MemFit trails consolidation-based rivals on multi-hop reasoning and beats the strongest baseline on LongMemEval-S by only 0.31 points, calling into question whether the efficiency-accuracy trade-off universally favors MemFit.
The append-only design is the paper's strongest conceptual contribution. By storing every turn verbatim and using segment summaries strictly as an index rather than a compressed replacement, MemFit avoids the irreversible information loss inherent in consolidation-based systems. The retrieval architecture successfully fuses complementary signals: BM25 for rare terms, cosine similarity for paraphrases, and a cross-encoder for final ordering. The efficiency improvements are substantial and quantified with rigor—81–87% fewer LLM calls and memory build times 3.9–52.6× faster than baselines. The system also demonstrates admirable backend-agnosticism, with one configuration performing well across proprietary and open-weight models.
The paper overstates its empirical dominance and understates its weaknesses. On LoCoMo multi-hop questions with GPT-4.1-mini, MemFit loses to SimpleMem by nearly two F1 points, yet the abstract labels the system 'state-of-the-art' without caveat. On LongMemEval-S the margin over LightMem is statistically fragile at 0.31 accuracy points. The ablation study is even more troubling: removing any single component—PRF, summary expansion, wide scan, or summaries themselves—often fails to reach statistical significance, suggesting overlapping redundancy rather than complementary design. Only the combined removal ('plain retrieval') produces a clear drop. The cost analysis also normalizes everything to GPT-4.1-mini API prices even when evaluating locally-hosted open-weight models, which misrepresents real deployment economics for self-hosted systems. Finally, the append-only design means storage grows linearly without bound, yet no scaling experiments probe when indexing or retrieval latency becomes prohibitive.
A deeper architectural concern is that the system's gains come partly by shifting work from write-time to read-time. The paper admits that MemFit's reader 'receives verbatim turns rather than distilled facts (2,210 vs. 1,015 input tokens per question), so full runs cost about the same ($1.53 vs. $1.46).' This trade-off is buried in the efficiency section and contradicts the impression of unalloyed cost savings.
The evidence supports the efficiency claims convincingly but only partially backs the accuracy narrative. Comparisons to prior work are procedurally fair—matching backbones and judges where possible—but the tiny margin on LongMemEval-S and the trailing multi-hop result undermine any claim of universal superiority. The choice of all-MiniLM-L6-v2 for embeddings and the ms-marco cross-encoder is conventional but not ablated; a stronger encoder could potentially alter the retrieval landscape and is never tested. The text-image fusion for multimodal retrieval is practical but not novel, relying entirely on caption quality, which is benchmark-provided for MemGallery and thus not stress-tested under noisy real-world captioning.
Reproducibility is moderately strong but has notable gaps. The authors release code, list hyperparameters ($\alpha=0.7$, $n_{scan}=60$, $k=10$, $w=1$), provide full prompts in Appendix F, and pin open-weight models to public Ollama tags. However, several practical details are underspecified: the pseudo-relevance feedback step 're-runs candidate generation' with rare terms but does not clarify whether the original query is retained or replaced; the TextTiling boundary threshold $\mu_d + \sigma_d/2$ lacks tie-breaking rules; and the multimodal captioning pipeline calls an unspecified MLLM for non-benchmark images. No random seed is reported, though greedy decoding is used. The biggest barrier to independent reproduction may be computational: LoCoMo runs required roughly 42 GPU-hours on four NVIDIA RTX PRO 6000 GPUs, a resource level not universally available.
Long-term memory systems for large language models (LLMs) have gained popularity for extending reasoning capabilities across applications. Current memory systems rely on LLM agents to organize and consolidate memory, resulting in costly, inefficient write operations. To address this limitation, we propose MemFit, a long-term memory system for conversational agents that reduces the cost and latency of memory operations. Unlike existing systems that rely on expensive LLM calls for memory construction or discard surface-level details through compression, MemFit stores each turn verbatim in an append-only store with near-instantaneous, LLM-free insertion, indexing turns with segment summaries rather than replacing them. Additionally, MemFit uses an LLM-free, multi-path retrieval strategy that combines lexical and semantic signals with cross-encoder reranking over caption- augmented episodes in both textual and multimodal settings. Empirical results on three widely used benchmarks, LoCoMo, MemGallery, and LongMemEval-S, show that MemFit achieves state-of-the-art performance while reducing memory construction time and cost several-fold, providing a scalable and efficient solution for persistent agentic memory.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.