WorksheetsBackpropagation calculus | Deep Learning Chapter 4
Total questions: 8
Worksheet time: 4mins
In a simplified neural network where each layer contains a single neuron, what parameters primarily determine the network's behavior?
Input values and activation functions
Weights and biases
Number of layers and neurons
Learning rate and epochs
For a single training example in a neural network, how is the cost function C0 typically defined, where a^(L) is the network's output and y is the desired output?
C0 = |a^(L) - y|
C0 = (a^(L) - y)^2
C0 = a^(L) * y
C0 = a^(L) + y
When analyzing the sensitivity of the cost function (C0) to a small change in a weight (w^(L)) in a computational graph, what does the term ∂C0/∂w^(L) represent?
The total change in the weight w^(L)
The ratio of a tiny change in C0 to a tiny change in w^(L)
The product of all weights in the network
The activation of the last neuron
In the context of backpropagation, the Chain Rule is applied to calculate the derivative of the cost function with respect to a weight. Which of the following correctly represents the Chain Rule for ∂C0/∂w^(L)?
(∂C0/∂a^(L)) * (∂a^(L)/∂z^(L)) * (∂z^(L)/∂w^(L))
(∂C0/∂w^(L)) + (∂a^(L)/∂z^(L)) + (∂z^(L)/∂w^(L))
(∂C0/∂z^(L)) * (∂z^(L)/∂a^(L)) * (∂a^(L)/∂w^(L))
(∂w^(L)/∂C0) * (∂z^(L)/∂a^(L)) * (∂a^(L)/∂C0)
In a single-neuron neural network, what is the partial derivative of the cost function C0 with respect to the weight w(L) of the output layer, given C0 = (a(L) - y)^2, a(L) = σ(z(L)), and z(L) = w(L)a(L-1) + b(L)?
2(a(L) - y)σ'(z(L))
a(L-1)σ'(z(L))2(a(L) - y)
σ'(z(L))2(a(L) - y)
a(L-1)2(a(L) - y)
When applying the chain rule to compute the derivative of the cost function C0 with respect to the bias b(L) in a single-neuron layer L, what is the value of the partial derivative ∂z(L)/∂b(L)?
0
1
a(L-1)
σ'(z(L))
For a neural network with multiple output neurons, which expression correctly defines the cost function C0, where a_j^(L) represents the activation of the j-th neuron in the output layer L, and y_j is the corresponding desired output?
C0 = (a_j^(L) - y_j)^2
C0 = Σ (a_j^(L) - y_j)
C0 = Σ (a_j^(L) - y_j)^2
C0 = (Σ a_j^(L) - Σ y_j)^2
In a multi-neuron neural network, why does the calculation of the derivative of the cost function C0 with respect to an activation a_k^(L-1) in layer L-1 involve a summation over neurons in layer L?
To account for the non-linearity of the activation function.
Because each neuron in layer L-1 contributes to the input of multiple neurons in layer L.
To simplify the overall gradient computation.
Because the bias terms are summed across layers.
