Reinforcement Learning and Deep RL Python Theory and Projects - Off Policy Versus On Policy

The video tutorial introduces two key terminologies in reinforcement learning: off-policy and on-policy. It explains that Q-learning follows an off-policy approach, where the learning agent derives the value function from another policy. In contrast, Sarsa uses an on-policy approach, learning from its current policy. The tutorial briefly touches on the mathematical equations for both methods but focuses on explaining the differences through Python code. The main distinction is that Q-learning seeks the maximum value from a new state, while Sarsa uses the value of a new action from its policy.

What is the main difference between off-policy and on-policy learning?

• On-policy does not use any policy.

• Off-policy uses another policy to learn.

• On-policy uses another policy to learn.

• Off-policy uses its own policy to learn.

In Q-learning, how is the value function learned?

• By using a random policy.

• By using the current policy.

• By using a different policy.

• By not using any policy.

Which technique is considered an on-policy method?

• Neither Q-learning nor Sarsa

• Q-learning

• Sarsa

• Both Q-learning and Sarsa

Why are the mathematical equations of Q-learning and Sarsa not discussed in detail?

• They are too simple.

• They are too complex for non-technical learners.

• They are not part of the course.

• They are not relevant.

In Q-learning, what is used to determine the next action?

• The minimum value of the new state.

• A random value from the Q table.

• The maximum value of the new state.

• The average value of the Q table.

What is the key difference in how Sarsa updates its value function compared to Q-learning?

• Sarsa does not update its value function.

• Sarsa uses the maximum value of the new state.

• Sarsa uses a random value from the Q table.

• Sarsa uses the value of the new action from the current policy.

In the context of Sarsa, what does the term 'new action' refer to?

• A random action from the Q table.

• The action with the lowest reward.

• The action selected by the current policy.

• The action with the highest reward.