ML techniques I week 2 session 2

A common rule of thumb says, “GRUs beat LSTMs on smaller data sets.” Which best explains why this can hold in practice?

• GRUs have fewer gates, so they regularise better with the same amount of data 

• GRUs converge faster because they use ReLU rather than tanh nonlinearities

• GRUs share weights across gates, reducing overfitting risk

• GRUs compute exact gradients, whereas LSTMs only approximate them

The core intuition behind adding gates to in LSTMs and GRUs is to 

• Ensure every hidden unit becomes non‑linear

• Speed up matrix multiplication on GPUs

• let the network dynamically choose how much past information to pass forward

• Remove the need for back‑propagation‑through‑time

The LSTM’s cell state is designed to act like a long‑term conveyor belt, while the hidden state mainly carries short‑term, view‑ready information

• True

• False

Intuitively, the reset gate in a GRU lets the model 

• scale all weights upward when gradients vanish

• decide whether to ignore the previous hidden state when constructing a new candidate

• lock the hidden state whenever it predicts a punctuation mark

• control how strongly the output is shown to the next layer

For sequential data (e.g., audio waveforms), why might a practitioner still reach for an RNNs over only temporal CNN layers, even if both can model context?

• GPUs cannot handle 1‑D convolutions efficiently

• RNNs never overfit as badly as CNNs

• CNNs require tanh activations, which are slower

• RNNs can, in principle, model arbitrarily long context with a fixed number of parameters