Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages
Crowdsourced pairwise evaluation has emerged as a scalable approach for assessing foundation models. However, applying it to Text to Speech(TTS) introduces high variance due to linguistic diversity and multidimensional nature of speech perception. We present a controlled multidimensional pairwise evaluation framework for multilingual TTS that combines linguistic control with perceptually grounded annotation. Using 5K+ native and code-mixed sentences across 10 Indic languages, we evaluate 7 state-of-the-art TTS systems and collect over 120K pairwise comparisons from over 1900 native raters. In addition to overall preference, raters provide judgments across 6 perceptual dimensions: intellig...
| Authors | Srija Anand, Ashwin Sankar, Ishvinder Sethi et al. |
|---|---|
| Primary category | cs.CL |
| Published | 23 April 2026 |
| Last updated at source | 23 June 2026 |
Connected in the knowledge graph
- ai4bharat/SpeechArenaBenchDataset · cited by
Models and datasets referencing this paper
- ai4bharat/SpeechArenaBenchDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.