Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts
We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in context that differs from what was stored parametrically during training. We document asymmetric biases: models tend to prefer in-context information about entities which appear in text, but prefer parametric information about entities which appear in images. We relate this asymmetry to the late representational alignment across modalities, showing that the longer processing time associated with resolving visual entities prevents the suppression of the model's usual factual recall mechanism, thus resulting in more parametric answers. Chain-of-tho...
| Authors | Athulith Paraselli, Etha Tianze Hua, Ellie Pavlick |
|---|---|
| Primary category | cs.CL |
| Published | 31 August 2026 |
| Last updated at source | 31 August 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.