Explore/Exploit Tradeoff

Can’t always choose the action with highest Q-value The Q-function is initially unreliable Need to explore until it is optimal Most common method: ε-greedy Take a random action in a small fraction of steps (ε) Decay ε over time There is some work on optimizing exploration Kearns & Singh, ML 1998 But people usually use this simple method