Exploring Reinforcement Learning Concepts

What is a Markov Decision Process (MDP)?

• A Markov Decision Process (MDP) is a type of neural network.

• A Markov Decision Process (MDP) is a method for sorting data in databases.

• A Markov Decision Process (MDP) is a statistical model for predicting weather patterns.

• A Markov Decision Process (MDP) is a framework for modeling decision-making with states, actions, transition probabilities, and rewards.

Define the Multi-Armed Bandit Problem.

• A strategy for maximizing profits in stock trading.

• A method for solving linear equations.

• A game where players compete to collect the most coins.

• The Multi-Armed Bandit Problem is a decision-making problem where a gambler must choose between multiple options (arms) to maximize rewards, balancing exploration and exploitation.

What does the epsilon in epsilon-greedy represent?

• Epsilon indicates the maximum reward achievable.

• Epsilon is the fixed value for the learning rate.

• Epsilon represents the total number of actions taken.

• Epsilon represents the probability of exploration in the epsilon-greedy algorithm.

Explain the concept of exploration vs. exploitation.

• Exploration is only about maximizing rewards.

• Exploitation involves taking risks without prior knowledge.

• Exploration and exploitation are the same process.

• Exploration is the act of seeking new information or options, while exploitation is the act of using known information or options to maximize rewards.

What is the purpose of the reward function in MDPs?

• The reward function only tracks the agent's performance over time.

• The reward function provides feedback to guide the agent's decision-making process.

• The reward function determines the optimal policy directly.

• The reward function is used to initialize the MDP.

How does the UCB (Upper Confidence Bound) algorithm work?

• The UCB algorithm only focuses on exploitation without considering exploration.

• The UCB algorithm randomly selects actions without any strategy.

• The UCB algorithm uses a fixed set of actions without updating based on performance.

• The UCB algorithm selects actions by maximizing the upper confidence bound, balancing exploration and exploitation.

What are the key components of a Markov Decision Process?

• States, Actions, Probability Distribution

• States, Actions, Policy

• States, Actions, Value Function

• States, Actions, Transition Model, Reward Function, Discount Factor

Describe the difference between deterministic and stochastic policies.

• Deterministic policies yield a single action per state, while stochastic policies yield a probability distribution of actions per state.

• Deterministic policies involve multiple actions per state, while stochastic policies involve a single action.

• Deterministic policies are random, while stochastic policies are fixed.

• Deterministic policies are only used in games, while stochastic policies are used in real-world scenarios.

What is the significance of the discount factor in MDPs?

• The discount factor only affects immediate rewards.

• The discount factor is used to calculate state transitions.

• The discount factor is significant as it balances immediate and future rewards in MDPs.

• The discount factor is irrelevant in MDPs.

How can the epsilon-greedy strategy be implemented?

• Choose actions randomly without any probability.

• Select actions based on a fixed strategy every time.

• Select actions randomly with probability epsilon and the best action with probability 1-epsilon.

• Always select the best action without randomness.