Reinforcement Learning Concepts Worksheet

Which statement best characterizes the interaction between the agent and environment when the environment is partially observable?

• The agent has direct and complete access to the true state at every time step

• The agent receives observations (O_t) that are correlated with the true state (S_t)

• The agent receives no feedback from the environment

• The agent only obtains reward signals without any state-related information

Which condition ensures that a process satisfies the Markov property?

• The next state depends on the entire history of past states

• The next state depends only on the action taken, ignoring the current state

• The next state depends only on the reward signal

• The next state depends only on the current state and not on past states

Which statement correctly describes an optimal policy?

• It maximizes the expected cumulative discounted reward over time

• It focuses only on maximizing immediate rewards

• It minimizes variability in rewards instead of maximizing returns

• It is always unique for all decision problems

In a stochastic grid world, if an intended action results in other movements with certain probabilities, what does this indicate?

• The policy is deterministic

• The system depends on past states

• Action outcomes are probabilistic, leading to multiple possible next states

• Rewards remain constant for every state-action pair

Which tuple correctly defines an MDP?

• (S, A, P, R, γ)

• (S, A, R, γ)

• (S, P, R, γ)

• (A, P, R, γ)

Which of the following is the correct tuple for a Markov Decision Process (MDP)?

• ((S, A, R))

• ((S, A, P, R, \gamma))

• ((S, P, R))

• ((A, P, R, \gamma))

What ensures convergence in iterative policy evaluation?

• The discount factor satisfies ( \gamma < 1 )

• The policy is stochastic

• Rewards are always zero

• The state space is infinite

What is the primary computational limitation of policy iteration?

• It converges very slowly

• It requires complete policy evaluation at every iteration

• It cannot handle stochastic environments

• It does not guarantee optimality

Why is dynamic programming considered computationally expensive?

• It requires full knowledge of transition probabilities and reward function

• It relies on random exploration

• It cannot incorporate reward signals

• It depends heavily on function approximation

What guarantees convergence of the Bellman expectation operator?

• Rewards are always zero

• Infinite action space

• Finite state space

• Discount factor satisfies ( \gamma \in [0,1) ) ensuring contraction