Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
An activation oracle is a language model trained to read another model's internal activations and describe them in natural language, for example to name a secret word the other model was trained to hide. Oracle answers carry no measure of confidence, which limits their use in auditing. We compare five ways of attaching a confidence score to an oracle's answer on this secret-word task, across four oracles from two model families (Qwen and Gemma, 8B to 27B parameters), at $6{,}000$ samples per method and oracle. The five methods rank the same way on all four oracles. Which method to use depends on one question: can the auditor list the possible answers in advance? If the auditor can, then h...
| Authors | Federico Torrielli, Peter Schneider-Kamp, Lukas Galke Poech |
|---|---|
| Primary category | cs.CL |
| Published | 25 May 2026 |
| Last updated at source | 3 August 2026 |
Popularity, last 4 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- EvilScript/Qwen3-8B-taboo-gold-r10-a16-e2-checkpointsModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 20 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.