FLARE: Diffusion for Hybrid Language Model
Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-latency deployment. Recent efficient-inference work has progressed along two axes: reducing the cost of each model invocation through efficient architectures, and reducing serial decoding steps through parallel generation. Hybrid attention backbones address the former, while diffusion language models (dLLMs) pursue the latter via iterative parallel denoising. Combining these advantages remains challenging: AR-to-dLLM conversion often fails to preserve seed-checkpoint capability, and hybrid-attention recurrent states and masking constraints make d...
| Authors | Yuchen Zhu, Jing Shi, Chongjian Ge et al. |
|---|---|
| Primary category | cs.LG |
| Published | 1 June 2026 |
| Last updated at source | 4 August 2026 |
Connected in the knowledge graph
- yuchen-zhu-zyc/HybridDiffusion-4BModel · cited by
- yuchen-zhu-zyc/HybridDiffusion-2BModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.