NLP_8_RNN

What is the purpose of language models in NLP?

• To determine the probability of a sequence of words
• b) To translate text from one language to another
• c) To generate semantic embeddings
• d) To generate random sentences
• e) To predict the next word given a sequence of words

What is the main limitation of bag of words techniques in encoding document vectors?

• a) They cannot handle large datasets
• b) They do not consider word order
• c) They require a lot of computational power
• d) They are always less accurate than other techniques
• e) They are usually very sparse

What is the intuition behind recurrence networks?

• a) The output is a function of the current input and the previous state
• b) The output is a function of the current input only
• c) The output is a function of the previous state only
• d) The output is a random function

What is the main benefit of using static word embeddings in language models?

• a) To improve the accuracy of language models
• b) To reduce the computational complexity of language models
• c) To handle large datasets more efficiently
• d) To generate random sentences
• e) Capture the contextual semantics

What is the primary advantage of using Recurrent Neural Networks (RNNs) over traditional feed-forward neural networks in natural language processing?

• a) RNNs have a simpler architecture
• b) RNNs can process variable-length sequences
• c) RNNs require less training data
• d) RNNs are computationally less expensive
• RNNs are usually more accurate than FFNN

In the context of an RNN cell, what does the term 'recurrence' refer to?

• a) Repeated application of the same function
• b) Feedback loop from output back to input
• c) Regular updating of network weights
• d) Reuse of the same layers for each input
• e) Feedback from the hidden state back to the input

In RNNs, what is the key significance of the hidden state (h_t) at each time step?

• a) It holds the output of the network at that step
• b) It is the input for the next layer of the network
• c) It represents the 'memory' of the network up to that point
• d) It is used to calculate the backpropagation error

What are the possible causes of the vanishing gradient problem in traditional RNNs?

• a) The repeated multiplication of gradients through the network layers
• b) The use of non-linear activation functions like sigmoid or tanh
• c) The limited memory capacity of RNNs
• d) The use of large learning rates during training
• e) The use of ReLU activation

Which of the following is NOT True about RNNs

• The output of the RNN is always the same as h(t)

• The dimension of the weight matrix depends of the shape of the input x(t)

• The main activation function is either ReLU or tanh

• RNNs are designed to handle sequential data

• The hidden state accumulates memory from previous inputs

Check the wrong statement :

• Training an RNN on very long sequences may easily result in vanishing Gradient problem

• In sequence labeling task, at each time step, the predicted probability distribution over the vocabulary is created using a softmax activation applied to the hidden layer output.

• In Language Modeling Task, the loss function at a given time step is calculated as the negative log probability of the correct target word in the vocabulary.

• The hidden state at the first time step is initialized to the embedding of the first word in the training sequence.

• In RNN Language Modeling Task, the objective is to maximize the probability of the correct word at the current time step, given the context seen so far.