Mastering Hyperparameter Tuning

What is the purpose of weight initialization in deep neural networks?

• To ensure all weights are set to zero for uniformity.

• The purpose of weight initialization in deep neural networks is to set the initial weights in a way that promotes effective learning and convergence.

• To randomly assign weights to neurons for diversity.

• To initialize weights based on the output of the previous layer.

How does choosing the appropriate activation function affect model performance?

• It only affects the model's training speed.

• It determines the model's input data format.

• The appropriate activation function improves learning efficiency and model capacity.

• It has no impact on model performance.

What is batch normalization and why is it used?

• Batch normalization is a technique to increase the learning rate of the model.

• Batch normalization is used to reduce the size of the training dataset.

• Batch normalization is a technique to normalize layer inputs in neural networks, improving training speed and stability.

• Batch normalization is a method to increase the number of layers in a neural network.

Explain the concept of gradient clipping and its benefits.

• Gradient clipping eliminates the need for regularization techniques.

• Gradient clipping helps stabilize training by preventing exploding gradients, leading to more reliable convergence and improved performance.

• Gradient clipping is used to enhance the model's complexity.

• Gradient clipping increases the learning rate for faster convergence.

What is the difference between L1 and L2 regularization?

• L1 regularization increases all weights equally; L2 regularization reduces the overall weight.

• L1 regularization is used for classification; L2 regularization is used for regression only.

• L1 regularization eliminates features; L2 regularization keeps all features intact.

• L1 regularization promotes sparsity; L2 regularization distributes weights more evenly.

How does dropout regularization help prevent overfitting?

• Dropout regularization eliminates the need for validation data.

• Dropout regularization only works with convolutional neural networks.

• Dropout regularization increases the number of neurons used during training.

• Dropout regularization helps prevent overfitting by randomly deactivating neurons during training, promoting robustness and reducing reliance on specific features.

What is early stopping and how does it improve training efficiency?

• Early stopping is a technique to enhance model complexity.

• Early stopping improves training efficiency by preventing overfitting and reducing unnecessary training time.

• Early stopping increases training time by allowing more epochs.

• Early stopping guarantees perfect model accuracy.

Describe adaptive gradient descent and its advantages over standard gradient descent.

• Adaptive gradient descent does not require any tuning of hyperparameters.

• Adaptive gradient descent adjusts learning rates per parameter, improving convergence speed and performance over standard gradient descent.

• Standard gradient descent is faster than adaptive gradient descent in all cases.

• Adaptive gradient descent uses a fixed learning rate for all parameters.

What is learning rate decay and why is it important?

• Learning rate decay is a method of reducing the learning rate over time to improve model training stability and performance.

• Learning rate decay is irrelevant to model performance and stability.

• Learning rate decay increases the learning rate to speed up training.

• Learning rate decay is a technique used to initialize weights in a model.

How does gradient descent with momentum work?

• Gradient descent with momentum improves convergence speed and stability by incorporating past gradients into the current update.

• It uses only the current gradient for updates.

• Momentum is applied only at the beginning of training.

• It eliminates the need for learning rates entirely.