Author: Arnaud Autef

<aside> 💡 In this presentation, we review a bandit learning paper from Daniel Russo: Simple Bayesian Algorithms for Best Arm Identification

</aside>

Context

• Sequential decision making problem.

• $k$ possible designs.

• Each design $i \in \{1,~...,~k\}$ has unknown parameter $\theta_i^*$

• At each time-step $n \in \mathbb{N}$

  1. experimenter chooses design $I_n \in \{1,~...,~k\}$
  2. experimenter observes a random outcome $Y_{I_n, n} \sim p(.|\theta_{I_n}^*)$

<aside> 💡 Identify which design $I^*$ leads to the best outcomes, in as few steps as possible.

</aside>

Practical Example

• Experimenter = Decision-makers at a pharmaceutical company

• $k$ designs = $k$ variants of a treatment to cure a disease

  • For each design $i$ = treatment $i$
    • $\theta_i^*$ = Success rate of treatment $i$ = probability of curing patients of the disease
    • Unknown at the start

• Optimal design $I^*$ = treatment variant with highest efficacy