DolphinBench: Mapping the Pareto Frontier of Agent Memory
Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one. Moreover, benchmarks rarely require anything beyond accuracy from submissions, allowing memory systems to make unreasonable cost/time tradeoffs to achieve higher scores. We present DolphinBench, a benchmark that evaluates memory directly through an agent's task completion. DolphinBench includes three knowledge-work personas with roughly 500k tokens of user messages per persona and evaluates agents on task...
| Authors | Soumil Rathi, Deshraj Yadav, Taranjeet Singh |
|---|---|
| Primary category | cs.CL |
| Published | 21 September 2026 |
| Last updated at source | 22 September 2026 |
Models and datasets referencing this paper
- mem0ai/dolphinbenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.