DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression
We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but are agnostic to the base model, allowing quantization noise to disproportionately corrupt the small-magnitude parameter deltas ($ΔW$) that encode post-training behavior -- an effect we analyze through the lens of quantization as implicit regularization. DAQ replaces reconstruction-based objectives with two delta-aware metrics -- Sign Preservation Rate and Cosine Similarity -- that directly optimize for directional fidelity of $ΔW$, requiring only the base and post-trained...
| Authors | Xiaoming Yu, Shize Tang, Guanghua Yu et al. |
|---|---|
| Primary category | cs.LG |
| Published | 20 March 2026 |
| Last updated at source | 20 March 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.