Information-Theoretic Foundations of Online Temperature Sampling
Repetitive, degenerate generation is a persistent failure of LLM decoding, and the standard remedies (temperature, nucleus, and top-$k$ sampling) are static heuristics. By Kac's lemma and the AEP, a stationary source takes time exponential in its entropy to repeat a block. We therefore maximize the entropy of the sampling distributions under one cross-entropy budget that keeps the output plausible under the model. Offline, the solution is temperature sampling, with the inverse temperature as the budget's Lagrange multiplier. \emph{Our contribution is the online problem}: the conditionals $p_1, p_2, \dots$ are revealed one at a time, and the sampling distribution $q_t$ must be chosen causa...
| Authors | Jin Sima, Nikolaos Papagiannis, Ananth Grama et al. |
|---|---|
| Primary category | cs.LG |
| Published | 10 October 2026 |
| Last updated at source | 10 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 14 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.