How Fine-Tuning Objectives Shape Layer-Wise Information in LLM Hallucination Detection
Under matched conditions, classifier-head fine-tuning (\textsc{cls}) paradigm consistently outperforms the LLM-as-Judge sequence-to-sequence (\textsc{s2s}) on binary hallucination detection. We trace this gap to the representations induced by the two objectives. Static analysis shows that \textsc{cls} learns a task-specific discriminative direction, pushing hidden states into a clear two-cluster geometry with stronger class separation and linearly decodable label information. By contrast, \textsc{s2s} relies on the pretrained language-modeling head to make decisions. This minimizes the pressure to reorganize representations, leaving the hidden states much closer to the pretrained backbone...
| Authors | Kaiyang Wan, Forrest Sheng Bao, Amin Ahmad et al. |
|---|---|
| Primary category | cs.AI |
| Published | 20 October 2026 |
| Last updated at source | 20 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 21 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.