Understanding Back Propagation

What is the primary goal of the Gradient Descent Optimization?

• To maximize the cost function.

• To minimize the cost function.

• To increase the number of iterations.

• To stabilize the learning rate.

How does the learning rate affect the convergence of the model?

• The learning rate only affects the model's accuracy, not convergence.

• The learning rate affects convergence by controlling the step size towards the minimum; too high can cause divergence, too low can slow down convergence.

• A higher learning rate always guarantees faster convergence.

• The learning rate has no effect on convergence.

What are the common types of activation functions used in neural networks?

• Exponential

• Sigmoid, Tanh, ReLU, Leaky ReLU, Softmax

• Polynomial

• Linear

Explain the difference between Mean Squared Error and Cross-Entropy in error calculation.

• Mean Squared Error is used for classification tasks, while Cross-Entropy is used for regression tasks.

• Mean Squared Error is always preferred over Cross-Entropy for all types of tasks.

• Mean Squared Error is used for regression tasks, while Cross-Entropy is used for classification tasks.

• Mean Squared Error measures probabilities, while Cross-Entropy measures distances.

What are the implications of poor weight initialization on training a neural network?

• Poor weight initialization can slow down training, cause convergence issues, and lead to suboptimal performance.

• Improved weight initialization can enhance training speed.

• Poor weight initialization guarantees optimal convergence.

• Weight initialization has no effect on model performance.

How can overfitting be identified in a machine learning model?

• The model shows consistent performance across all datasets.

• Overfitting can be identified by a significant performance gap between training and validation/test datasets.

• The model has a high accuracy on the training dataset only.

• The model performs equally well on both training and validation datasets.

What are some common techniques used for regularization in neural networks?

• Batch normalization

• Activation functions

• Gradient descent

• L1 regularization, L2 regularization, dropout, early stopping, data augmentation.

How does the choice of activation function impact the training process?

• It determines the learning rate of the model.

• The choice of activation function impacts convergence speed, gradient flow, and the model's ability to learn complex patterns.

• It affects the number of layers in the network.

• It has no impact on the model's performance.

What role does the learning rate play in the Gradient Descent algorithm?

• The learning rate adjusts the momentum in the Gradient Descent algorithm.

• The learning rate determines the number of iterations in Gradient Descent.

• The learning rate sets the maximum number of features to consider.

• The learning rate controls the step size in the Gradient Descent algorithm.

Why is it important to calculate the error accurately during training?

• It reduces the training time significantly.

• It ensures effective optimization and improves model performance.

• It has no impact on the final model accuracy.

• It simplifies the data preprocessing steps.