RNN&Transformers Quiz

What is the main challenge associated with training vanilla RNNs on long sequences?

• Exploding gradients

• Vanishing gradients

• Both exploding and vanishing gradients

• None of the above

What is the main limitation of using one-hot encoding for representing words in an RNN?

• a) One-hot encoding does not capture semantic relationships between words.

• b) One-hot encoding is inefficient for large vocabularies.

• c) One-hot encoding can lead to exploding gradients during training.

• Both a) and b).

What is the key idea behind Recurrent Neural Networks (RNNs)?

• RNNs have an internal state that is updated as a sequence is processed.

• RNNs use a fixed set of parameters for each time step.

• RNNs can only process sequential data.

• RNNs are feed-forward neural networks.

What is the purpose of the "output gate" in an LSTM?

• It controls what information goes into the output of the LSTM cell.

• It determines how much information goes through the cell state.

• It decides what information is added to the cell state.

• It updates the cell state.

What is the purpose of the "forget gate" in an LSTM (Long Short-Term Memory) network?

• It determines how much information goes through the cell state.

• It decides what information is added to the cell state.

• It controls what goes into the output.

• It updates the cell state.

What is the purpose of the "attention mechanism" in sequence-to-sequence models?

• To improve the model's ability to capture long-term dependencies.

• To reduce the computational complexity of the model.

• To allow the model to focus on different parts of the input sequence when generating the output sequence.

• To prevent vanishing gradients during training.

In the context of sequence to sequence (seq2seq) models, what is the purpose of the "encoder" component?

• To generate the output sequence from the input sequence.

• To encode the input sequence into a fixed-size vector representation.

• To decode the fixed-size vector representation into the output sequence.

• To combine the input and output sequences into a single sequence.

What is the purpose of truncated backpropagation through time (TBPTT) in RNNs?

• To reduce memory requirements for long sequences.

• To speed up training by skipping some time steps.

• To improve the model's ability to learn long-term dependencies.

• To prevent exploding gradients.

In the context of language modeling, what is the task that an RNN is trying to solve?

• Predict the next word in a sequence given the previous words.

• Translate a sequence of words from one language to another.

• Generate a caption for a given image.

• Classify a sequence of words into a predefined category.

What is the main advantage of using LSTMs (Long Short-Term Memory) over vanilla RNNs?

• LSTMs are faster to train than vanilla RNNs.

• LSTMs can better capture long-term dependencies in sequences.

• LSTMs are more memory-efficient than vanilla RNNs.

• LSTMs can only be used for sequence generation tasks, while vanilla RNNs can be used for both sequence generation and sequence classification.