Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer sampling steps, process the full sequence at every step, and require many passes over the dataset to reach good quality. We introduce Next-Concept Modeling (NCM), a two-stage, coarse-to-fine framework: a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting all codes within a level in parallel. At matched parameter scales, NCM achieves better generative perplexity than the...
| Authors | Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi et al. |
|---|---|
| Primary category | cs.CL |
| Published | 9 October 2026 |
| Last updated at source | 9 October 2026 |
Popularity, last 3 days
0 now (-1 since 2026-10-15). Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- hazyresearch/ncm-datasetsDataset · cited by
Recent changes
- 16 Oct 2026 Description: Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer → Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer
- 16 Oct 2026 Topics added cs.CL, cs.AI, cs.LG removed hf-daily-papers, diffusion language models, parallel sampling, multiscale residual vq-vae, discrete codes, transformer, generative perplexity, function evaluations
Changes detected by Silytics between refreshes of the source.
Models and datasets referencing this paper
- hazyresearch/ncm-datasetsDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.