MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models
Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time, outdated knowledge can pose safety risks. Existing evaluations of knowledge integration focus on factual recall, offering limited insight into whether newly integrated knowledge is actually usable. Our benchmark MedKIT (Medical Knowledge Integration and Transfer) provides a granular evaluation of how models integrate and apply knowledge under realistic sequences of clinical updates. Each instance corresponds to a factual update derived from clinical evidence, paired with targeted probes that assess transfer across lexical variation, relat...
| Authors | Lukas Thede, Yash Kumar Atri, David Chen et al. |
|---|---|
| Primary category | cs.CL |
| Published | 29 September 2026 |
| Last updated at source | 29 September 2026 |
Connected in the knowledge graph
- bethgelab/MedKITDataset · cited by
Models and datasets referencing this paper
- bethgelab/MedKITDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.