Gradient Decent

What is gradient descent?

• 1. A machine learning algorithm used for classification tasks.

• 1. An optimization algorithm used to minimize a function by iteratively adjusting the parameters.

• 1. A supervised learning technique used for regression problems.

• A statistical approach for clustering data points into groups.

How does gradient descent work?

• 1. It tries random parameter values and selects the one that yields the lowest loss.

• 1. It uses matrix operations to minimize the loss function.

• 1. It calculates the gradient of the loss function with respect to the parameters and updates the parameters in the opposite direction.

• 1. It calculates the gradient of the loss function with respect to the parameters and updates the parameters in the same direction.

What is the purpose of gradient descent?

• • To find the global minimum of a function.

• • To maximize the accuracy of a machine learning model.

• To solve linear equations

• • To find the local minimum of a function.

What is the role of the learning rate in gradient descent?

• It determines the speed at which the model learns and converges to the optimal solution.

• It defines the size of each step taken during the optimization process.

• It influences how quickly the model adapts to changes in the input data.

• All of the above.

What is the difference between batch gradient descent and stochastic gradient descent?

(select two best)

• • In batch gradient descent, all data points are considered for each parameter update, while in stochastic gradient descent, only one data point is used.

• • Batch gradient descent is faster but less accurate compared to stochastic gradient descent.

• • Stochastic gradient descent is suitable for large datasets, while batch gradient descent is preferred for small datasets.

• • Batch gradient descent more accurate compared to stochastic gradient descent.

What is momentum-based gradient descent?

• A variant of gradient descent that introduces a momentum term to accelerate convergence and dampen oscillations.

• A technique that adjusts the learning rate dynamically based on the magnitude of the gradients.

• An optimization algorithm that computes the gradients of the loss function using only a subset of the training data.

• A method for regularizing neural networks to prevent overfitting.

There are how many types of Gradient Descent?

• 1

• 2

• 3

• 4