Sumi: Open Uniform Diffusion Language Model from Scratch
Diffusion models have become a promising alternative to autoregressive models. Among these, uniform diffusion language models (UDLMs) permit any token to be updated at any step, in principle enabling more flexible generation. However, no UDLM has yet been pretrained from scratch at both large parameter scale and large token budget. Both autoregressive modeling and masked diffusion modeling already have capable models at scale that the community can study and build on; uniform diffusion has none. A scratch-pretrained UDLM at scale would provide a clean reference point for studying scaling behavior, generation dynamics, controllability, and trade-offs against established autoregressive and ...
| Authors | Mengyu Ye, Keito Kudo, Wataru Ikeda et al. |
|---|---|
| Primary category | cs.CL |
| Published | 17 June 2026 |
| Last updated at source | 17 June 2026 |
Popularity, last 2 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- tohoku-nlp/sumi-7bModel · cited by
Models and datasets referencing this paper
- tohoku-nlp/sumi-7bModel
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.