Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness
Large language models are characterized by three key properties: capability, alignment, and faithfulness. Prior work studies the tradeoffs between capability and alignment, and between capability and faithfulness, but a third tension remains underexplored: the alignment-faithfulness conflict. We show that aligned models systematically deviate from their inputs on unsafe or sensitive content without disclosing the modification, a failure mode we call alignment-induced unfaithfulness (AIU). Unlike capability-driven unfaithfulness, which comes from errors in knowledge or reasoning, this is induced by post-training mechanisms that override adherence to the input. We introduce FaithConflict, a...
| Authors | Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur et al. |
|---|---|
| Primary category | cs.CL |
| Published | 30 September 2026 |
| Last updated at source | 30 September 2026 |
Models and datasets referencing this paper
- PardisSzah/FaithConflictDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.