A population of \(N\) users will be shown version A or B. First \(n\) users per arm are used to test, the rest receive the version that looks better (test and roll). With Gaussian priors \(\mu_j\sim\mathcal N(m,\tau^2)\) on the mean reward of each arm and noise \(\sigma\), the expected total reward is \(Nm+(N-2n)\,\tau^2/(\sqrt\pi\sqrt{\tau^2+\sigma^2/n})\). Testing more makes the choice more reliable but uses up users; the best size \(n^*\) has a closed form. Change the prior, the noise and the population and see where the optimum goes.
The plot shows the expected reward above the no-information value \(Nm\) (the second term of the formula). The error rate is \(\tfrac12-\tfrac1\pi\arctan(\tau\sqrt n/\sigma)\), the probability that the arm with the smaller true mean is deployed. The simulation draws \(\mu_0,\mu_1\) from the prior, the two sample means, deploys the arm with the larger sample mean, and averages the total reward over 20,000 experiments.
code/ch34_decision_making.py (ab_nstar, ab_expected_reward, ab_error_rate, ab_regret, simulate_ab).