Interactive explainer

Compare contextual bandits interactively

Two situations favour different actions. Explore what a global policy misses and how separate posteriors learn.

Interactive

Compare contextual bandits interactively

The simulation computes Thompson sampling with two separate context models and with one shared model.

What is calculated

Two situations favour different actions. In context 1, A succeeds with probability 80% and B with 20%. Context 2 reverses these values. Both policies start with uniform beta priors. The contextual policy updates only the model for the observed situation, while the global policy pools all observations.

The lines show regret: expected loss relative to a policy that already knows the best action in each context. The table counts actions actually selected. Successes are random outcomes and should not be confused with regret. In each round, both policies share the same potential outcomes for each action if they choose that action. Their random draws for action selection remain separate.

Three experiments

  1. Equal frequency: Keep the context share at 50% and increase the number of decisions. Watch how A and B are distributed within each context.
  2. A rare situation: Set context 1 to 90%. Compare the number of observations in the second context with the total sample size.
  3. A new environment: Enable the halfway shift. Both pairs of success rates swap, while the models retain all previous observations.

Reading the result

The fixed seed of 42 makes every setting reproducible. A single run demonstrates a mechanism, not a universal advantage. The shift highlights that ordinary posteriors have no automatic forgetting. Unknown or continuous contexts would require different models, and real processes may deliver feedback much later than this example.

The article on Contextual bandits explains the update, provides a worked numerical example and discusses practical limitations.

From NyxAI

smartlytics is the learning hub of NyxAI GmbH.

Explore the methods we work with, explained openly, with sources and calculations you can reproduce.

Decision Core

NyxAI is developing a Decision Core for Decision Intelligence that combines optimisation, simulation, statistics and reinforcement learning: compare options, make uncertainty visible and learn from outcomes, with people in control.

AI workflows at NyxAI

DeepSquare

Explainable chess AI with human playing styles: NyxAI’s testing ground for understandable decisions under time pressure.