Python for Deep Learning - Build Neural Networks in Python - What is Stochastic Gradient Descent?

The video tutorial discusses the limitations of gradient descent in nonconvex functions due to multiple local minima. It introduces stochastic gradient descent (SGD) as an alternative, highlighting its randomness and efficiency in training time. The tutorial also explains mini-batch gradient descent, which uses a subset of data, offering a balance between SGD and traditional gradient descent.

Why might gradient descent struggle with nonconvex functions?

• It requires too much computation.

• It only works with linear functions.

• It can get stuck in local minima.

• It converges too quickly.

What is a key characteristic of stochastic gradient descent?

• It introduces randomness by using a single input per iteration.

• It uses the entire dataset for each iteration.

• It is deterministic and follows a fixed path.

• It always finds the global minimum.

How does the path taken by stochastic gradient descent compare to traditional gradient descent?

• It is smoother and more predictable.

• It always follows the same path.

• It is noisier but faster.

• It is slower and less efficient.

What is mini-batch gradient descent?

• A method that uses a single data point per iteration.

• A technique that does not involve randomness.

• A technique that uses the entire dataset at once.

• A variant of SGD that uses a subset of data points.

What is the main advantage of using stochastic gradient descent?

• It guarantees finding the global minimum.

• It requires no data preprocessing.

• It reduces training time significantly.

• It eliminates the need for a learning rate.