Deep Learning Quiz 1

1. The McCulloch-Pitts neuron, a foundational concept in neural networks, is characterized by which of the following?

• It is characterized by binary inputs, a threshold function, and binary outputs.
• It uses continuous inputs and outputs.
• It operates solely on analog signals.
• It requires multiple layers for processing.

2. What is the primary purpose of the backpropagation algorithm in training a Feed Forward Neural Network?

• To update the weights of the neural network to minimize the loss function.
• To initialize the neural network's architecture.
• To increase the complexity of the model.
• To generate random weights for the network.

3. Which issue is Batch Normalization primarily designed to address during the training of deep neural networks?

• Vanishing gradients
• Overfitting
• Exploding gradients
• Internal covariate shift

4. In the context of dimensionality reduction, how does a basic, undercomplete autoencoder's approach relate to Principal Component Analysis (PCA)?

• An undercomplete autoencoder only reduces noise, unlike PCA which captures variance.
• PCA uses deep learning techniques, while autoencoders do not.
• An undercomplete autoencoder learns a compressed representation like PCA, focusing on capturing variance in the data.
• An undercomplete autoencoder requires labeled data, whereas PCA does not.

5. Which gradient descent optimization algorithm combines the concepts of Momentum and RMSProp?

• Adam
• Adagrad
• SGD
• Nesterov

6. A Denoising Autoencoder is trained to reconstruct the original, clean input after it has been fed a corrupted version of that input. What is the primary benefit of this approach?

• It reduces the complexity of the model by simplifying the input data.
• It focuses solely on learning from clean inputs without any noise.
• It eliminates the need for any preprocessing of the input data.
• It improves the model's robustness and generalization by learning from corrupted inputs.

7. What is a key difference between a standard autoencoder and a Variational Autoencoder (VAE)?

• A VAE models the latent space probabilistically, unlike a standard autoencoder.
• A standard autoencoder generates data from a fixed distribution.
• A VAE requires more layers than a standard autoencoder.
• A standard autoencoder uses a deterministic approach to model the latent space.

8. In a deep feedforward network, if the weights are all initialized to the same value (e.g., zero or a constant), what problem will occur during training?

• The network will learn faster due to uniform weights.
• All neurons will specialize in different tasks effectively.
• The training process will converge to a global minimum immediately.
• The network will fail to learn effectively due to symmetry, as all neurons will behave identically.

9. Which of the following is a common form of dataset augmentation for image data?

• Image blurring
• Image rotation and flipping
• Color saturation adjustment
• Noise addition

10. What distinguishes Nesterov Accelerated Gradient (NAG) from classical Momentum-based Gradient Descent?

• NAG does not utilize any momentum term in its calculations.
• NAG uses a lookahead mechanism to compute gradients at a predicted future position, unlike classical Momentum which computes gradients at the current position.
• NAG is slower than classical Momentum due to its complexity.
• NAG computes gradients using only the current position, similar to classical Momentum.