From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
Large language models produce fluent, confident, factually wrong output. Existing taxonomies classify these failures by output type -- intrinsic versus extrinsic, faithfulness versus factuality -- but say nothing about which computational component produced a given failure. We ask what would be required to attribute an individual hallucination to a specific component of the decoder-only stack. We treat three components -- self-attention's associative retrieval, the maximum-likelihood pretraining objective, and autoregressive commitment under exposure bias -- as candidate failure surfaces, justify their separability rather than assuming it, and specify an attribution procedure requiring on...
| Authors | Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem |
|---|---|
| Primary category | cs.CL |
| Published | 29 April 2026 |
| Last updated at source | 4 September 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.