When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
Visualizations help communicate data insights, but deceptive data representations can distort their interpretation and propagate misinformation. While recent Vision Language Models (VLMs) perform well on many chart understanding tasks, their ability to detect misleading visualizations, especially when deception arises from subtle reasoning errors in captions, remains poorly understood. Here, we evaluate VLMs on misleading visualization-caption pairs grounded in a fine-grained taxonomy of reasoning errors (e.g., Cherry-picking, Causal inference) and visualization design errors (e.g., Truncated axis, Dual axis, inappropriate encodings). To this end, we develop a benchmark that combines real...
| Authors | Harsh Nishant Lalai, Raj Sanjay Shah, Hanspeter Pfister et al. |
|---|---|
| Primary category | cs.CV |
| Published | 23 March 2026 |
| Last updated at source | 19 April 2026 |
Models and datasets referencing this paper
- HarvardVCG/MisVisBenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.