When Does Streaming Tool Use Help? Characterizing Tool-Intent Stabilization in Streaming Retrieval-Augmented Generation
Streaming Retrieval-Augmented Generation (Streaming RAG) hides tool latency by issuing retrieval queries in parallel with the user's still-arriving input, before the utterance is complete. Speculation can only help, though, when the correct query becomes determinable before the user stops speaking or typing -- a property of the query, not the system. We name and measure this property, tool-intent stabilization: the point in the input stream at which a speculative query's retrieval converges on the answer-bearing result. On the CRAG benchmark (1371 validation questions) we (i) characterize how stabilization is distributed across queries; (ii) derive a model-agnostic bound H on the share of...
| Authors | Elroy Galbraith |
|---|---|
| Primary category | cs.CL |
| Published | 18 June 2026 |
| Last updated at source | 19 June 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.