mishig/autoresearch-optimizer-findings
Ablating Optimizer Ideas Around MuonAdamW for Transformer Pretraining Overview 27 optimizer experiments on a 50M-parameter transformer, systematically testing whether anything can beat a well-tuned MuonAdamW baseline. We tried curvature-aware momentum, spectral boosting via power iteration, dual-timescale EMAs, variance-adaptive scaling, sign-coherence weighting, and more — drawing from recent papers (NAMO, AdEMAMix, Muon-VS, ROOT) and first-principles linear… See the full description on the dataset page: https://huggingface.co/datasets/mishig/autoresearch-optimizer-findings.
| Author / organization | mishig |
|---|---|
| Downloads | 208 (as reported by Hugging Face) |
| Likes | 9 |
| Created | 9 April 2026 |
| Last updated at source | 10 April 2026 |
Popularity, last 3 days
208 now (-6 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 17 Oct 2026 Description: Ablating Optimizer Ideas Around MuonAdamW for Transformer Pretraining Overview 27 optimizer experiments on a 50M-parameter transformer, systematically testing whether anything can beat a well-tuned MuonAdamW baseline. We tried curvature-awa → Ablating Optimizer Ideas Around MuonAdamW for Transformer Pretraining Overview 27 optimizer experiments on a 50M-parameter transformer, systematically testing whether anything can beat a well-tuned MuonAdamW baseline. We tried curvature-awa
Changes detected by Silytics between refreshes of the source.
More from mishig
- mishig/sample_imagesDataset
- mishig/thanos_picking_power_gemDataset
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.