Multi-View Latent MoE: Letting Every Router Shape Expert Computation
Sparse mixture-of-experts (MoE) models expand parameter capacity by activating a small subset of experts per token. Expert selection directly affects model capabilities under a fixed active-expert budget. Dense evaluation of all experts reveals weak agreement between single-router rankings and expert-utility evaluation rankings, especially in middle and deep layers. Richer signals for selection and computation incur additional cost: selection based on internal expert activations requires preliminary computation, while processing multiple full representations separately through the same expert repeats its computation. We propose Multi-View Latent MoE to improve expert selection with multip...
| Authors | Zelin Zhao, Junyuan Shang, Tianchi Ma et al. |
|---|---|
| Primary category | cs.CL |
| Published | 29 October 2026 |
| Last updated at source | 2 November 2026 |
Popularity, last 12 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- baidu/ERNIE-5.5-Flash-Base-BF16Model · cited by
- baidu/ERNIE-5.5-Flash-BaseModel · cited by
Recent changes
- 03 Nov 2026 Name: Multi-View MoE: Letting Every Router Shape Expert Computation → Multi-View Latent MoE: Letting Every Router Shape Expert Computation
- 03 Nov 2026 Description: Sparse mixture-of-experts (MoE) models expand parameter capacity by activating a small subset of experts per token. Expert selection directly affects model capabilities under a fixed active-expert budget. Dense evaluation of all experts rev → Sparse mixture-of-experts (MoE) models expand parameter capacity by activating a small subset of experts per token. Expert selection directly affects model capabilities under a fixed active-expert budget. Dense evaluation of all experts rev
Changes detected by Silytics between refreshes of the source.
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 11 November 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.