Autoencoders and RL Concepts Quiz

Autoencoders require labeled training data so that the decoder learns to reconstruct the correct target output.

• True

• False

Answer explanation

False. Autoencoders do not require labeled training data; they learn to reconstruct input data by minimizing the difference between the input and output, making them unsupervised learning models.

If an autoencoder’s bottleneck is too large, the network may fail to learn meaningful compressed features.

• True

• False

Answer explanation

True. If the bottleneck of an autoencoder is too large, it can retain too much information, preventing the model from learning to compress data effectively and extract meaningful features.

Self-Organizing Maps preserve topological relationships, meaning nearby neurons often represent similar input patterns.

• True

• False

Answer explanation

True. Self-Organizing Maps (SOMs) maintain topological relationships, meaning that neurons that are close together in the map typically respond to similar input patterns, preserving the structure of the input data.

In a SOM, only the Best Matching Unit updates; all other neurons remain unchanged.

• True

• False

Answer explanation

In a Self-Organizing Map (SOM), not only the Best Matching Unit (BMU) updates, but also its neighbors can be adjusted based on a neighborhood function. Therefore, the statement is False.

Denoising autoencoders add noise to the input but still train to reconstruct the clean version.

• True

• False

Answer explanation

True. Denoising autoencoders intentionally add noise to the input data during training, but they learn to reconstruct the original, clean version of the input, making the statement accurate.

Q-Learning requires knowing the environment’s transition probabilities to update Q-values.

• True

• False

Answer explanation

The statement is False because Q-Learning is a model-free reinforcement learning algorithm. It updates Q-values based on the agent's experiences without needing to know the environment's transition probabilities.

In RL, a policy that never explores is guaranteed to discover the globally optimal behavior.

• True

• False

Answer explanation

The statement is false because a policy that never explores will only exploit known information, potentially missing better actions. Exploration is essential in reinforcement learning to discover optimal behaviors.

A higher discount factor γ makes the agent value future rewards more strongly.

• True

• False

Answer explanation

True. A higher discount factor γ increases the present value of future rewards, making the agent prioritize and value them more strongly compared to lower discount factors.

In sparse reward environments, even heavy exploration may fail because the agent rarely encounters useful feedback.

• True

• False

Answer explanation

True. In sparse reward environments, agents receive infrequent feedback, making it difficult to learn effectively even with extensive exploration. This lack of useful feedback hinders the agent's ability to improve its performance.

SOM training is essentially identical to k-means since both update only the winning unit per sample.

• True

• False

Answer explanation

The statement is false because SOM (Self-Organizing Map) updates multiple units based on their distance to the input, not just the winning unit, unlike k-means which only updates the centroid of the cluster for the winning unit.