mi1_13_RL-Q

Did you attempt the last exercise sheet?

• Yes, I finally did

• No )-:

Which Q-value is highest for the tile in the center (the smurf is the agent)?

• Q(x, →)

• Q(x, ←)

• Q(x, ↓)

• Q(x, ↑)

An MDP implies...

• I know all possible states beforehand

• model-based learning

• model-free evaluation

• policy iteration

Which provides the most direct way for extracting the optimal policy π*?

• V* = argmax V^π

• Q* = argmax Q^π

• neither

• what does the * mean?

SARSA stands for

• some small constant

• state action reward state action

• such a small reward so awful

• StAte Reward StAte

The optimal policy π* is unique (T/F)

• True

• False

With Q-values, the optimal policy π* becomes unique (T/F)

• True

• False

"contracting" means

• converging

• monotonically decreasing

• falling inside the neighborhood of the true value

• fluctuating around the true value

For a learning rate η>0, TD Learning...

• contracts and converges

• contracts but does not converge

• does not contract but converges

• does not contract, does not converge

Which describes "off policy"?

• watching someone else play a game and judging them

• I play a game and judge my own actions

• I can only see someone's actions but not the states.

• The policy diverges, it goes off-course