Single-Round Vector RAG vs an LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research Corpus
We preregistered a comparison of two ways to help an LLM answer questions over a small research corpus: single-round Vector RAG and an LLM-compiled markdown wiki browsed by a tool-using agent. Both answered the same 13 questions over 24 papers with the same answer model, scored by two blinded LLM judges. The three preregistered predictions came out one weakly supported, one supported, and one refuted. The wiki scored much better at connecting findings across papers, but its organization advantage fell below the registered threshold once both judges were combined. RAG met the registered test on single-fact groundedness, though the result was judge-sensitive. The wiki was cheaper to build b...
| Authors | Theodore O. Cochran |
|---|---|
| Primary category | cs.CL |
| Published | 18 May 2026 |
| Last updated at source | 2 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.