IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answering
AI systems have achieved remarkable success in processing text and relational data, yet visual document processing remains relatively underexplored. Whereas traditional systems require OCR transcriptions to convert these visual documents into text and metadata, recent advances in multimodal foundation models offer retrieval and generation directly from document images. This raises a key question: How do image-based systems compare to established text-based methods? We introduce IRPAPERS, a benchmark of 3,230 pages from 166 scientific papers, with both an image and an OCR transcription for each page. Using 180 needle-in-the-haystack questions, we compare image- and text-based retrieval and...
| Authors | Connor Shorten, Augustas Skaburskas, Daniel M. Jones et al. |
|---|---|
| Primary category | cs.IR |
| Published | 5 February 2026 |
| Last updated at source | 5 February 2026 |
Popularity, last 2 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- HughieHu/drpixel-benchDataset · cited by
- mteb/IRPapersRetrievalDataset · cited by
Models and datasets referencing this paper
- HughieHu/drpixel-benchDataset
- mteb/IRPapersRetrievalDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 19 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.