A.X K2 Technical Report
We introduce A.X K2, a 688B-parameter Mixture-of-Experts (MoE) language model trained from scratch as a high-performance foundation for \emph{agentic} applications. Trained on approximately 8.5T tokens---fewer than its predecessor, A.X K1---on a smaller but higher-quality mixture with substantially expanded agentic and software-engineering data, it nonetheless improves over A.X K1 across the board, by over 30 percentage points on some benchmarks, reflecting large gains in token efficiency. To support long contexts efficiently, we introduce Sparse Gated Attention (SGA), which combines sparse attention with gated attention, and adopt Gated Norm (GN) to stabilize large-scale training. SGA is...
| Authors | Cheolseung Baek, Dhammiko Arya, Eunki Kim et al. |
|---|---|
| Primary category | cs.AI |
| Published | 31 August 2026 |
| Last updated at source | 31 August 2026 |
Connected in the knowledge graph
- skt/A.X-K2Model · cited by
Models and datasets referencing this paper
- skt/A.X-K2Model
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.