Learned Structure Shows Up Earlier in the Task's Own Basis: A Controlled Look at a Synchronisation Precursor in Grokking
Analyses of grokking usually read hidden activations in one basis, most often the Fourier basis modulo $p$. We ask how much the answer to "has the network organised yet?" depends on that choice. The measure FSD$_{\rm gen}$ averages each unit's activation over inputs with the same answer and counts how many units share the same strongest irreducible representation. For modular multiplication we read one answer-averaged profile in three orders: discrete-logarithm order, in which the characters of the multiplicative group become Fourier modes; natural order; and random orders. Across 60 networks at $p\in\{59,97,113\}$, the discrete-log reading reached a fixed threshold before both other read...
| Authors | Achyuthan Sivasankar |
|---|---|
| Primary category | cs.LG |
| Published | 11 June 2026 |
| Last updated at source | 11 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.