Neural Network Optimization Quiz

What is the main goal of optimizing neural networks?

• To increase the number of layers

• To minimize a loss function

• To maximize the dataset size

• To simplify the architecture

Which optimization algorithm is known for being simple and easy to understand?

• Gradient Descent

• Momentum

• Adam

• Stochastic Gradient Descent

What is a disadvantage of Stochastic Gradient Descent (SGD)?

• It is too simple

• It can introduce noise

• It is computationally expensive

• It requires no tuning

What does Mini-batch Gradient Descent balance?

• Noise and convergence

• Speed and accuracy

• Efficiency and stability

• Complexity and simplicity

What is the purpose of the momentum coefficient in the Momentum algorithm?

• To decrease the batch size

• To reduce oscillations

• To eliminate noise

• To increase the learning rate

Which algorithm converges faster than Momentum?

• Adagrad

• RMSprop

• Nesterov Accelerated Gradient

• Adam

What is a key feature of Adagrad?

• It adjusts learning rates per parameter

• It is suitable for large datasets

• It uses a constant learning rate

• It requires manual tuning

What is a disadvantage of RMSprop?

• It is not suitable for sparse data

• It has a vanishing learning rate

• It is too complex to implement

• It needs learning rate tuning

What does Adam combine the benefits of?

• Momentum and Nesterov

• RMSprop and SGD

• Momentum and Adagrad

• Adagrad and RMSprop

What is a disadvantage of Adam?

• It requires no tuning

• It is suitable for all datasets

• It is easy to implement

• It uses more memory and computation