Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought

The strongest element is the evidence-side diagnostics. An LLM judge and deterministic checks show that coverage—supplying sufficient evidence—is the dominant bottleneck across systems, while conditional utilization (accuracy given coverage) varies far less. Raw-RAG leads coverage on both MemoryAgentBench and MEME, and it preserves executable dependency paths best (62.34% conditional path coverage versus 9.62% for MemU). A second strong point is the efficiency analysis: Raw-RAG completes ingestion plus retrieval in 305.91 seconds on a MEME slice, $5.1\times$ faster than the nearest competitor and $30.4\times$ faster than EverMemOS. The audit of Mem0 mutations is also telling: "Replacing UPDATE with append-only ADD and DELETE with NONE raises LoCoMo accuracy from 67.36% to 69.42%, without reducing any category score." (Section 1, Table 2)

Several limitations temper the central claim. First, the comparisons treat agentic systems largely as black boxes with their native configurations, whereas Raw-RAG is explicitly tuned for this study (cross-encoder reranking, 256-token chunks, top-15 selection). Prior RAG baselines in the literature were often weaker because they used coarse chunks or no reranking, so the gap may reflect retrieval quality rather than a fundamental irrelevance of memory management. Second, the benchmarks remain static, offline QA tasks over completed histories; they do not test incremental online ingestion, user-specific deletion or privacy constraints, or multimodal settings where raw-history storage explodes. Third, absolute performance on hard dynamic tasks is still low: on MEME cascade, Raw-RAG reaches only 4.27% with GPT-4o-mini (Table 4), and even oracle raw-evidence does not close the gap to Full-context. The authors acknowledge this: "Our evidence is limited to offline conversation on public benchmarks; it does not establish performance for interactive, multimodal, or broader operational settings." (Section 6.1) Finally, the framing risks overgeneralization from conversation QA to all agentic memory scenarios.

Another concern is the fairness of the baseline comparison. The authors note that prior evaluations reported RAG trailing agentic memory because of coarse retrieval units, restricted retrieval depth, or absent reranking (Appendix E). This means the paper is partly comparing against weakened RAG baselines, and the conclusion that agentic memory is unnecessary could instead be reframed as: a well-tuned RAG baseline is necessary before claiming agentic memory gains. As they write, "These limitations leave open whether the reported gaps reflect a need for agentic memory systems or weaknesses in the chosen retrieval baselines." (Section 1)

Within the evaluated conversation benchmarks, the empirical evidence supports the ranking claims. Raw-RAG leads in six of seven settings by margins of 0.13–5.53 points, and the coverage/utilization decomposition gives a credible mechanistic explanation. Comparisons to related work are fair in noting that prior RAG baselines were under-specified, though the paper could more directly test whether simply adding reranking to those baselines recovers most of the gap. One tension is that MEME itself was designed to expose dependency-reasoning failures: the MEME authors found that "we find that all systems collapse on dependency reasoning under the default configuration (Cascade: 3%, Absence: 1% in average accuracy)" (Jung et al., Abstract). Bao et al. report similar collapse for most systems, with Raw-RAG only modestly ahead on cascades (4.27% GPT-4o-mini, 3.66% DeepSeek), suggesting raw retrieval helps but does not solve the underlying reasoning bottleneck. The complete-store oracle (Section 5.3) is a clever control, showing that "The two source-preserving systems recover performance closest to the raw-evidence control, whereas access to the complete transformed stores leaves substantially larger deficits," which supports the information-loss thesis.

Reproduction is aided by detailed appendices: Appendix B gives chunking, retrieval, and reranking specifications; Appendix D describes execution conditions; and Appendix G supplies prompt templates. Hyperparameters are mostly reported (256-token windows, 32-token overlap, 128 dense candidates, top-15 reranker, Qwen3-Embedding-4B / Cohere Rerank-v3.5). However, no code repository or official data-release URL is stated in the main text, and index-implementation details (flat vs. approximate search, embedding batch sizes) are relegated to appendices that may not be fully visible in all renders. The reliance on proprietary API endpoints (GPT-4o-mini, GPT-4.1-mini) and specific commercial rerankers means reproduction carries cost and version-dependency risks. Without an announced open-source release, independent reproduction is possible but not frictionless. The authors do provide some transparency: "Appendix B specifies the full pipeline," (Section 3) and "Appendix D details execution conditions." (Section 4.3)