Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs
Self-interpretation methods prompt language models to describe their own internal states, but remain unreliable due to hyperparameter sensitivity. We show that training lightweight adapters on interpretability artifacts, while keeping the LM entirely frozen, yields reliable self-interpretation across tasks and model families. A scalar affine adapter with just $d_\text{model}+1$ parameters suffices: trained adapters generate sparse autoencoder feature labels that outperform the training labels themselves (70% vs 50% generation scoring at 70B scale), identify topics with 94% recall@1 versus 1% for untrained baselines, and decode bridge entities in multi-hop reasoning that appear in neither ...
| Authors | Keenan Pepper, Alex McKenzie, Florin Pop et al. |
|---|---|
| Primary category | cs.CL |
| Published | 10 February 2026 |
| Last updated at source | 2 June 2026 |
Popularity, last 3 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Models and datasets referencing this paper
- bahnassy/robust-honesty-evalDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 20 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.