MAPLE: MoE Adaptive Plug-and-play Layer-wise Expert allocation
Sparsely-activated Mixture-of-Experts (MoE) Transformers universally fix the same number of routed experts across all layers, a convention that ignores the well-documented heterogeneity in layer-wise redundancy. We demonstrate that this uniformity is systematically suboptimal and propose MAPLE, a plug-and-play framework that reallocates the routed-expert budget heterogeneously across layers of any pretrained MoE LLM, without modifying weights or requiring retraining. Our core contribution is a closed-form sensitivity-guided allocation: we probe each layer's response to variation in expert count, quantify sensitivity using three measures, and derive an analytically optimal budget assignmen...
| Authors | Lie Li, Wen Li, Junxiao Shen et al. |
|---|---|
| Primary category | cs.LG |
| Published | 15 August 2026 |
| Last updated at source | 25 August 2026 |
Connected in the knowledge graph
- Davd-b01/qwen3.8-flash-next-40b-prune-researchModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.