Understanding Dropout in Neural Networks

What is the primary effect of dropout on a neural network during training?

• It increases the overall size of the neural network.

• It randomly deactivates units, effectively creating smaller networks.

• It permanently removes specific layers from the network.

• It reduces the number of input features to the network.

How does dropout encourage a neural network to spread out its weights?

• By increasing the learning rate for individual features.

• By making units unable to rely on any single input feature.

• By adding a fixed L1 penalty to all weights.

• By reducing the total number of layers in the network.

When implementing dropout, how should the "keep_prob" parameter be adjusted for layers more susceptible to overfitting?

• It should be set to 1.0 to retain all units.

• It should be set to a higher value, such as 0.9.

• It should be set to a lower value, such as 0.5.

• It should be kept constant across all layers.

What is a significant challenge when debugging gradient descent in a neural network that uses dropout?

• The network's training speed significantly decreases.

• The cost function (J) no longer monotonically decreases, making it harder to track optimization.

• The number of parameters to tune increases exponentially.

• It becomes impossible to calculate the gradients accurately.