Tracing distinguishability through transformer processing with stochastic LayerNorm
Representational similarity is foundational to analyses of deep networks, yet distances between point-valued representations are not intrinsically tied to downstream function: nearby states may produce different behaviors, while distant states may behave similarly. We instead give representations volume, turning similarity into statistical distinguishability. Overlapping stochastic representations necessarily induce overlapping downstream distributions, grounding latent comparison in model function and bringing it under information-theoretic tools such as the data-processing inequality. We realize this idea in pretrained transformers through a light-touch modification to LayerNorm: at eac...
| Authors | Kieran Murphy |
|---|---|
| Primary category | cs.LG |
| Published | 31 August 2026 |
| Last updated at source | 31 August 2026 |
Connected in the knowledge graph
- murphysics999/stoch_norm_gpt2_smallModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.