Interactive explainer
Compare contextual bandits interactively
Two situations favour different actions. Explore what a global policy misses and how separate posteriors learn.
Compare contextual bandits interactively
The simulation computes Thompson sampling with two separate context models and with one shared model.
What is calculated
Two situations favour different actions. In context 1, A succeeds with probability 80% and B with 20%. Context 2 reverses these values. Both policies start with uniform beta priors. The contextual policy updates only the model for the observed situation, while the global policy pools all observations.
The lines show regret: expected loss relative to a policy that already knows the best action in each context. The table counts actions actually selected. Successes are random outcomes and should not be confused with regret. In each round, both policies share the same potential outcomes for each action if they choose that action. Their random draws for action selection remain separate.
Three experiments
- Equal frequency: Keep the context share at 50% and increase the number of decisions. Watch how A and B are distributed within each context.
- A rare situation: Set context 1 to 90%. Compare the number of observations in the second context with the total sample size.
- A new environment: Enable the halfway shift. Both pairs of success rates swap, while the models retain all previous observations.
Reading the result
The fixed seed of 42 makes every setting reproducible. A single run demonstrates a mechanism, not a universal advantage. The shift highlights that ordinary posteriors have no automatic forgetting. Unknown or continuous contexts would require different models, and real processes may deliver feedback much later than this example.
The article on Contextual bandits explains the update, provides a worked numerical example and discusses practical limitations.