AjayRangoji/AR
AR (Autoregressive 4B) — Full Pretraining Codebase & Infrastructure Pretraining codebase, cluster orchestration scripts, data pipeline, and evaluation harness for AR-4B, a 4.02B-parameter decoder-only transformer pretrained from scratch on ~325B tokens across up to 64x NVIDIA H100 GPUs. Pretrained Weights & Model Card: AjayRangoji/AR-4B-base Architecture: 4.02B parameters, 36 layers, hidden size 2,560, GQA (32/8 heads, head dim 128), SwiGLU MLP (inner size 9,728), QK-norm… See the full description on the dataset page: https://huggingface.co/datasets/AjayRangoji/AR.
| Author / organization | AjayRangoji |
|---|---|
| License (as declared) | apache-2.0 |
| Downloads | 0 (as reported by Hugging Face) |
| Likes | 0 |
| Created | 26 October 2026 |
| Last updated at source | 26 October 2026 |
Recent changes
- 26 Oct 2026 Description: none → AR (Autoregressive 4B) — Full Pretraining Codebase & Infrastructure Pretraining codebase, cluster orchestration scripts, data pipeline, and evaluation harness for AR-4B, a 4.02B-parameter decoder-only transformer pretrained from scratch on
- 26 Oct 2026 Topics removed en
- 26 Oct 2026 Topics added en
- 26 Oct 2026 Description: none → AR (Autoregressive 4B) — Full Pretraining Codebase & Infrastructure Pretraining codebase, cluster orchestration scripts, data pipeline, and evaluation harness for AR-4B, a 4.02B-parameter decoder-only transformer pretrained from scratch on
- 26 Oct 2026 Topics removed en
- 26 Oct 2026 Topics added en
- 26 Oct 2026 Description: none → AR (Autoregressive 4B) — Full Pretraining Codebase & Infrastructure Pretraining codebase, cluster orchestration scripts, data pipeline, and evaluation harness for AR-4B, a 4.02B-parameter decoder-only transformer pretrained from scratch on
- 26 Oct 2026 Topics removed en
- 26 Oct 2026 Topics added en
- 26 Oct 2026 Description: none → AR (Autoregressive 4B) — Full Pretraining Codebase & Infrastructure Pretraining codebase, cluster orchestration scripts, data pipeline, and evaluation harness for AR-4B, a 4.02B-parameter decoder-only transformer pretrained from scratch on
- 26 Oct 2026 Topics removed en
- 26 Oct 2026 Topics added en
Changes detected by Silytics between refreshes of the source.
More from AjayRangoji
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 26 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.