CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing
Continual adaptation of large language models (LLMs) to evolving task distributions while mitigating catastrophic forgetting remains an open problem. Existing strategies often struggle to scale or lack a robust mechanism for zero-shot task-to-expert assignment. We introduce CaLLM, a framework that resolves these tensions by combining metric-based meta-learning with modular parameter isolation via lightweight adapters. First, CaLLM projects each input batch into a task-agnostic embedding space to find its nearest prototype. It then routes the batch to the specific adapter specialized for that data distribution. This formulation provides (i)~controllable cross-task interference by design, (...
| Authors | Bilge Celik, Anna Vettoruzzo, Joaquin Vanschoren |
|---|---|
| Primary category | cs.LG |
| Published | 27 October 2026 |
| Last updated at source | 27 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 28 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.