Mellum2 Technical Report
We present Mellum 2, an open-weight 12B-parameter Mixture-of-Experts (MoE) language model with 2.5B active parameters per token. Mellum 2 is a general-purpose language model specialized in software engineering, spanning code generation and editing, debugging, multi-step reasoning, tool use and function calling, agentic coding, and conversational programming assistance, and it is the successor to the completion-focused 4B dense Mellum model. The architecture builds on the Mixture-of-Experts (64 experts, 8 active) and combines Grouped-Query Attention with 4 KV heads, Sliding Window Attention on three of every four layers, and a single Multi-Token Prediction head that doubles as both an auxi...
| Authors | Marko Kojic, Ivan Bondyrev, Aral de Moor et al. |
|---|---|
| Primary category | cs.CL |
| Published | 29 May 2026 |
| Last updated at source | 29 May 2026 |
Models and datasets referencing this paper
- JetBrains/Mellum2-12B-A2.5B-BaseModel
- JetBrains/Mellum2-12B-A2.5B-InstructModel
- JetBrains/Mellum2-12B-A2.5B-Thinking-SFTModel
- JetBrains/Mellum2.1-12B-A2.5B-ThinkingModel
- JetBrains/Mellum2-12B-A2.5B-ThinkingModel
- cyankiwi/Mellum2-12B-A2.5B-Thinking-AWQ-INT4Model
- niteg/Mellum2.1-12B-A2.5B-Thinking-oQ5eModel
- Mungert/Mellum2-12B-A2.5B-Instruct-GGUFModel
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.