Deep learning Batch 1

In the context of tensors, what is the order of a matrix?

• A) 0th-order

• B) 1st-order

• C) 2nd-order

• D) 3rd-order

Which of the following is true about the probability density function (PDF) for continuous variables?

• A) The PDF must satisfy \( p(x) \leq 1 \)

• B) The integral of the PDF over its domain must equal 1

• C) The PDF can take negative values

• D) The PDF is only defined for discrete variables

What is the primary difference between Batch Gradient Descent and Stochastic Gradient Descent (SGD)?

• A) SGD uses the entire dataset for each update, while Batch GD uses a single example

• B) SGD uses a single example for each update, while Batch GD uses the entire dataset

• C) SGD is slower but more accurate than Batch GD

• D) SGD is only used for convex functions

Which of the following is a key disadvantage of Stochastic Gradient Descent (SGD)?

• A) It requires large memory to compute gradients

• B) It has high variance in parameter updates

• C) It converges slower than Batch Gradient Descent

• D) It cannot escape local minima

What is the main purpose of the learning rate in gradient-based optimization?

• A) To control the speed of convergence

• B) To increase the number of iterations

• C) To reduce the loss function directly

• D) To increase the model complexity

Which of the following is true about L1 regularization?

• A) It penalizes the square of the weights

• B) It is less effective than L2 regularization

• C) It is also known as weight decay

• D) It can reduce some weights to exactly zero

What is the primary purpose of dropout in neural networks?

• A) To reduce the number of layers in the network

• B) To increase the learning rate

• C) To randomly remove nodes during training to prevent overfitting

• D) To reduce the number of parameters in the model

Which of the following is a common technique to reduce overfitting in deep learning?

• A) Increasing the number of layers

• B) Decreasing the amount of training data

• C) Using dropout

• D) Increasing the learning rate

What is the main challenge of using a very high learning rate in gradient descent?

• A) It may cause the algorithm to overshoot the minimum

• B) It will always converge to the global minimum

• C) It will reduce the number of iterations required

• D) It will increase the model's bias

Which of the following is true about the Adam optimizer?

• A) It uses a fixed learning rate for all parameters

• B) It combines the benefits of Adagrad and RMSprop

• C) It is less efficient than SGD

• D) It does not use momentum