Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision
Speech Large Language Models (LLMs) that understand and follow instructions in many languages are useful for real-world interaction, but are difficult to train with supervised fine-tuning, requiring large, task-specific speech corpora. While recent distillation-based approaches train performant English-only Speech LLMs using only annotated ASR data by aligning text and speech using only a lightweight projector, these models under-perform when scaled to multilingual settings due to language interference in the shared projector. We address this by introducing language-aware distillation using a query bank and a gating network that selects or mixes query tokens using a Q-Former projector. Ou...
| Authors | Shreyas Gopal, Donghang Wu, Ashutosh Anshul et al. |
|---|---|
| Primary category | cs.CL |
| Published | 7 March 2026 |
| Last updated at source | 24 July 2026 |
Connected in the knowledge graph
- shreyasg97/Audio-MLQADataset · cited by
Models and datasets referencing this paper
- shreyasg97/Audio-MLQADataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.