[Back to Videos](/videos)

# Thompson Sampling: Why Your A/B Test Wastes Half Its Traffic

77

Retrieval & Search Research

Mixpeek Team

July 20, 2026

## Summary

A fixed 50/50 A/B test sends half of every cohort to the inferior variant until the test ends. Multi-armed bandits reframe the decision as allocation under uncertainty, and Thompson sampling (1933) solves it in one loop: keep a belief distribution per option, sample once from each every round, play the highest sample, update with the result. Uncertain options explore themselves, winners take over automatically, and regret is provably near-optimal (Agrawal & Goyal 2012). Netflix picks homescreen artwork with contextual bandits; ranking under uncertain relevance is the same math. Explore: mixpeek.com/glossary/relevance-ranking

thompson-samplingmulti-armed-banditsab-testingexperimentationrankingmachine-learning

## About this video

A fixed 50/50 A/B test sends half of every cohort to the inferior variant until the test ends. Multi-armed bandits reframe the decision as allocation under uncertainty, and Thompson sampling (1933) solves it in one loop: keep a belief distribution per option, sample once from each every round, play the highest sample, update with the result. Uncertain options explore themselves, winners take over automatically, and regret is provably near-optimal (Agrawal & Goyal 2012). Netflix picks homescreen artwork with contextual bandits; ranking under uncertain relevance is the same math. Explore: mixpeek.com/glossary/relevance-ranking

## Frequently Asked Questions

What is Thompson sampling?

Why do bandits beat fixed A/B tests?