Deep Learning CNN Convolutional Neural Networks with Python - Weight Initialization

The video tutorial explains gradient descent and its role in finding the global minimum of a loss function. It discusses the differences between convex and non-convex functions in neural networks and the challenges posed by vanishing and exploding gradients. The tutorial also covers weight initialization strategies to improve learning efficiency and concludes with a summary and a preview of future topics like learning rate.

What is the primary goal of gradient descent in the context of a convex loss function?

• To avoid any minima

• To maximize the loss function

• To find the global minimum

• To find the local minimum

Why is weight initialization important in neural networks with non-convex loss functions?

• It determines the learning rate

• It eliminates the need for activation functions

• It affects the convergence to local minima

• It ensures the loss function is convex

What problem arises when using sigmoid activation functions with zero weight initialization?

• The weights become too large

• The activations and gradients become zero

• The learning rate becomes negative

• The network overfits the data

Which activation function is preferred to mitigate the vanishing gradient problem?

• Softmax

• Sigmoid

• Tanh

• ReLU

How does the layer size affect the variance of the normal distribution used for weight initialization?

• Variance is always constant

• Layer size does not affect variance

• Smaller layers require larger variance

• Larger layers require larger variance

What is the main advantage of using normal distribution for weight initialization?

• It guarantees finding the global minimum

• It ensures weights are always positive

• It eliminates the need for activation functions

• It speeds up the learning process

What is the primary focus when finding a minimum in neural networks?

• Finding a feasible minimum

• Maximizing the loss function

• Avoiding any minima

• Finding the global minimum