Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazing
Multi-modal large language models (MLLMs) have advanced general-purpose video understanding but struggle with long, high-resolution videos -- they process every pixel equally in their vision transformers (ViTs) or LLMs despite significant spatiotemporal redundancy. We introduce AutoGaze, a lightweight module that removes redundant patches before processed by a ViT or an MLLM. Trained with next-token prediction and reinforcement learning, AutoGaze autoregressively selects a minimal set of multi-scale patches that can reconstruct the video within a user-specified error threshold, eliminating redundancy while preserving information. Empirically, AutoGaze reduces visual tokens by 4x-100x and ...
| Authors | Baifeng Shi, Stephanie Fu, Long Lian et al. |
|---|---|
| Primary category | cs.CV |
| Published | 12 March 2026 |
| Last updated at source | 12 March 2026 |
Models and datasets referencing this paper
- nvidia/NVILA-8B-HD-VideoModel
- nvidia/AutoGazeModel
- bfshi/HLVidDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.