Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
Audio large language models (Audio LLMs) exhibit systematic failures in transcribing code-switching speech despite strong multilingual capabilities. Focusing on English-Mandarin, we identify three failure modes: language omission, translation-instead-of-transcription, and hallucination. We apply Direct Preference Optimization (DPO) to align models, constructing preference pairs in which chosen responses preserve mixed-language content while rejected responses mimic failure patterns. Training three Audio LLMs on 100K pairs (570 hours), we observe consistent behavioral shifts: models learn to preserve language composition rather than translating when prompted for transcription. This alignme...
| Authors | Trung Nguyen Quang, Cheng Yi Lewis Won, Minh Duc Pham et al. |
|---|---|
| Primary category | cs.CL |
| Published | 13 May 2026 |
| Last updated at source | 1 October 2026 |
Models and datasets referencing this paper
- ngqtrung/emilia-dpoDataset
- ngqtrung/cs-dialogue-dpoDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.