FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference
Language models are increasingly adopting smaller architectures optimized for consumer devices. In this setting, inference efficiency is the primary constraint. Meanwhile, vocabulary sizes continue to grow rapidly, making the classification head a critical bottleneck that accounts for up to 60\% of model parameters, and 50\% of inference compute. We introduce FlashHead, the first efficient drop-in replacement for the dense classification head that is training-free and hardware-friendly. FlashHead builds on principles from information retrieval, reframing that computation at the output head as a retrieval problem rather than a dense classification over the full vocabulary. FlashHead introd...
| Authors | Wilhelm Tranheden, Shahnawaz Ahmed, Devdatt Dubhashi et al. |
|---|---|
| Primary category | cs.LG |
| Published | 15 March 2026 |
| Last updated at source | 15 March 2026 |
Connected in the knowledge graph
- embedl/Qwen3.5-27B-FlashHeadModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.