C5M1

Suppose your training examples are sentences (sequences of words). Which of the following refers to the j^th word in the i^th training example?

• x^(i)<j>

• x^<i>(j)

• x^(j)<i>

• x^<j>(i)

• T_x = T_y

• T_x < T_y

• T_x > T_y

• T_x = 1

• Speech recognition (input an audio clip and output a transcript)

• Sentiment classification (input a piece of text and output a 0/1 to denote positive or negative sentiment)

• Image classification (input an image and output a label)

• Gender recognition from speech (input an audio clip and output a label indicating the speaker’s gender)

• Estimating P(y^<1>, y^<2>, ...., y^<t-1>)

• Estimating P(y^<1>)

• • Estimating P(y^<t> | y^<1>, y^<2>, ...., y^<t-1>)

• Estimating P(y^<t> | y^<1>, y^<2>, ...., y^<t>)

• (i) Use the probabilities output by the RNN to pick the highest probability word for that time-step as y^{<t>. (ii) Then pass the ground-truth word from the training set to the next time-step.}

• (i) Use the probabilities output by the RNN to randomly sample a chosen word for that time-step as y^<t>. (ii) Then pass the ground-truth word from the training set to the next time-step.

• (i) Use the probabilities output by the RNN to pick the highest probability word for that time-step as y^<t>. (ii) Then pass this selected word to the next time-step.

• (i) Use the probabilities output by the RNN to randomly sample a chosen word for that time-step as y^<t>. (ii) Then pass this selected word to the next time-step.

You are training an RNN, and find that your weights and activations are all taking on the value of NaN (“Not a Number”). Which of these is the most likely cause of this problem?

• Vanishing gradient problem

• • Exploding gradient problem.

• • ReLU activation function g(.) used to compute g(z), where z is too large.

• Sigmoid activation function g(.) used to compute g(z), where z is too large.

Suppose you are training a LSTM. You have a 10000 word vocabulary, and are using an LSTM with 100-dimensional activations a^<t>. What is the dimension of Γ_u at each time step?

• 1

• 100

• 300

• 10000

• Alice’s model (removing Γ_u), because if Γ_r ≈0 for a timestep, the gradient can propagate back through that timestep without much decay.

• Alice’s model (removing Γ_u), because if Γ_r ≈1 for a timestep, the gradient can propagate back through that timestep without much decay.

• Betty’s model (removing Γ_r), because if Γ_u ≈0 for a timestep, the gradient can propagate back through that timestep without much decay.

• Betty’s model (removing Γ_u), because if Γ_r ≈1 for a timestep, the gradient can propagate back through that timestep without much decay.

• Γ_u and 1- Γ_u

• Γ_u and Γ_r

• 1- Γ_u and Γ_u

• Γ_r and Γ_u

You have a pet dog whose mood is heavily dependent on the current and past few days’ weather. You’ve collected data for the past 365 days on the weather, which you represent as a sequence as x^/<1>, …, x^/<365>. You’ve also collected data on your dog’s mood, which you represent as y^/<1>, …, y^/<365>. You’d like to build a model to map from x→y. Should you use a Unidirectional RNN or Bidirectional RNN for this problem?

• Bidirectional RNN, because this allows the prediction of mood on day t to take into account more information.

• Bidirectional RNN, because this allows backpropagation to compute more accurate gradients.

• Unidirectional RNN, because the value of y^/ depends only on x^/<1>,.....,x^/ but not on x^/<t+1>,.....,x^/<365>

• Unidirectional RNN, because the value of y^/depends only on x^/ , and not other days’ weather.