Statistically-Lossless Quantization of Large Language Models
Model quantization has become essential for efficient large language model deployment, yet existing approaches present clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but are lossy, while lossless techniques preserve fidelity but lack inference acceleration. This paper explores the middle ground of statistically-lossless compression, examining three complementary aspects of what losslessness means for quantized LLMs. First, task-lossless compression preserves zero-shot benchmark accuracy within natural sampling variance and is achievable at aggressive bitwidths. Second, we formalize the stricter notion of distribution-lossless compression, requiring the quanti...
| Authors | Michael Helcig, Eldar Kurtic, Dan Alistarh |
|---|---|
| Primary category | cs.LG |
| Published | 4 May 2026 |
| Last updated at source | 9 August 2026 |
Popularity, last 5 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 21 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.