LLM Agents Already Know When to Call Tools -- Even Without Reasoning
Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reaso...
| Authors | Chung-En Sun, Linbo Liu, Ge Yan et al. |
|---|---|
| Primary category | cs.CL |
| Published | 10 May 2026 |
| Last updated at source | 20 May 2026 |
Models and datasets referencing this paper
- cesun/When2ToolDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.