Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs
Popular ASR test sets adopt inconsistent conventions for numbers, disfluencies, entities, and casing, while standard normalizers erase the format distinctions users care about. Current benchmarks therefore cannot measure whether a model follows user preferences for output style. We introduce PreferenceASR, a test set evaluating ASR systems on their ability to follow natural-language preference instructions across four categories: normalization, entities, disfluencies, and case. Built from seven open-source corpora via a two-stage LLM-assisted pipeline with human verification, it is evaluated with a preference-aware normalizer that selectively skips steps matching the active instruction. B...
| Authors | Nithin Rao Koluguri, Sasha Meister, Nikolay Karpov et al. |
|---|---|
| Primary category | cs.CL |
| Published | 28 June 2026 |
| Last updated at source | 28 June 2026 |
Connected in the knowledge graph
- nvidia/preference-asr-benchDataset · cited by
Models and datasets referencing this paper
- nvidia/preference-asr-benchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.