Quiz on Loss and Activation Functions in Deep Learning

What is a loss function used for in deep learning?

• To visualize the training process

• To optimize the model's architecture

• To quantify how well a model's predictions match the true labels

• To increase the model's accuracy

Which loss function is commonly used in regression problems?

• Cross-Entropy Loss

• Mean Squared Error (MSE)

• Hinge Loss

• Sparse Categorical Cross-Entropy

What does the Mean Absolute Error (MAE) measure?

• The dissimilarity between true labels and predicted probabilities

• The average of the absolute differences between true and predicted values

• The divergence between two probability distributions

• The average of the squares of the errors

Which loss function is primarily used in Support Vector Machines (SVMs)?

• Mean Squared Error

• Kullback-Leibler Divergence Loss

• Hinge Loss

• Softmax

What is the main purpose of activation functions in neural networks?

• To simplify the training process

• To introduce non-linearity into the network

• To reduce the number of parameters

• To increase the model's complexity

Which activation function maps input values between 0 and 1?

• Sigmoid

• Softmax

• Tanh

• ReLU

What issue does the ReLU activation function suffer from?

• Overfitting

• Dying ReLU problem

• Vanishing gradients

• Underfitting

What does the Softmax function do?

• Converts raw scores into probabilities that sum to 1

• Maps input values to a range between -1 and 1

• Measures the divergence between two distributions

• Calculates the mean of the errors

Which activation function is known for its exponential growth for negative inputs?

• Leaky ReLU

• ELU

• Swish

• GELU

What is the formula for Mean Squared Error (MSE)?

• D_KL(P||Q) = ∑ P(i) log(P(i)/Q(i))

• L = -1/n ∑ [yi log(ŷi) + (1 - yi) log(1 - ŷi)]

• MSE = 1/n ∑ (yi - ŷi)²

• MAE = 1/n ∑ |yi - ŷi|