OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive audio and video from a user and return text. The user's query is embedded in the audio and video, without a separate text question, external captioning, or speech recognition. Direct audio-visual input reduces external latency and computation while preserving perceptual cues. However, research on OmniVChat faces two constraints: data availability and evaluation. Recordings of people using their own devices are scarce. Furthermore, good replies often depend on multimodal context and can be phrased in many ways, mak...
| Authors | Haolin He, Yunfei Chu, Qi Chen et al. |
|---|---|
| Primary category | eess.AS |
| Published | 18 September 2026 |
| Last updated at source | 28 September 2026 |
Connected in the knowledge graph
- Harland/OmniVChatDataset · cited by
Models and datasets referencing this paper
- Harland/OmniVChatDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.