Parser Transitions Already Reveal Structural Responsibility: KV Persistence for Structured Generation
LLM agents use structured generation for JSON, SQL, and function calls. Constrained decoding ensures valid formats, but KV compression can discard history needed for correct function and argument selection. Structural responsibility can diverge from model saliency: importance scores can overlook generated tokens' structural roles. Parser transitions already computed for legality encode these roles, including required arguments and structural boundaries, providing a free retention signal. We introduce PASK (Parser-Aware Structural KV Persistence), which reuses this signal for KV persistence by layer group without online importance estimation. Task-error sensitivity sets protection floors, ...
| Authors | Linze Wu, Xinrui Chen |
|---|---|
| Primary category | cs.LG |
| Published | 28 August 2026 |
| Last updated at source | 13 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.