Variance Reduction for Heavy-Tailed Monetization Metrics in Ranking Experiments via Post-Stratification
Online evaluation of ranking and retrieval systems often relies on downstream monetization metrics such as app revenue or creator earnings. These metrics are typically heavy-tailed, with a small fraction of users dominating both mean and variance, leading to low statistical power and unreliable conclusions in A/B experiments -- especially under limited traffic. We present a practical framework for variance reduction in online experiments by combining post-stratification with CUPED. Our approach leverages pre-experiment covariates to improve the sensitivity of monetization experiments without requiring additional traffic. Deployed at ShareChat across ranking-driven monetization experiments...
| Authors | Neeti Pokharna, Olivier Jeunen, Yatharth Saraf et al. |
|---|---|
| Primary category | cs.LG |
| Published | 2 June 2026 |
| Last updated at source | 2 June 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.