shawnghu/nanogpt-fullbatch

Full-batch gradient descent on the NanoGPT speedrun model Everything from a set of experiments training the NanoGPT speedrun world-record model (record #360, "ANVIL2") with exact full-batch gradients, compared against ordinary small-batch training on exactly the same data. The code is at github.com/shawnghu/nanogpt-fullbatch. The result in one table All runs use the same model and fixed objective. "Training-sample loss" is the loss on a fixed 10M-token sample… See the full description on the dataset page: https://huggingface.co/datasets/shawnghu/nanogpt-fullbatch.

<iframe src="https://silytics.si/embed/item/d12afc3c-0a4a-4c6c-8db7-b332417f228c" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.