ML B2 CH5

What is the purpose of a language model in NLP?

• Translate text between languages
• Assign probability to a sequence of words
• Generate one-hot encodings
• Filter stop words from text

Answer explanation

Language models predict the likelihood of a given sequence of words, enabling natural language understanding.

What is a key limitation of CBOW as a language model?

• It overfits easily
• It ignores the order of words in context
• It needs GPU to run
• It can't process numerical data

Answer explanation

CBOW treats context as a bag of words and does not consider the order, which limits its language modeling ability.

What does RNN stand for?

• Random Neural Node
• Recurrent Neural Network
• Relational Network Node
• Regular Numeric Network

Answer explanation

RNN stands for Recurrent Neural Network, which is designed to handle sequence data like language.

How does RNN differ from traditional neural networks?

• It processes all data at once
• It forgets past inputs
• It maintains a hidden state across time steps
• It only works on images

Answer explanation

RNNs pass a hidden state from one time step to the next, allowing them to retain memory of past inputs.

What is Truncated BPTT in RNN training?

• A way to randomly drop neurons
• A method to skip over rare data points
• A technique to limit the range of backpropagation through time
• A normalization step

Answer explanation

Truncated BPTT reduces the time span for backpropagation, making training more efficient and stable.

What is the purpose of using mini-batches in RNN training?

• To shuffle input data randomly
• To reduce memory usage and improve training speed
• To increase perplexity
• To remove outliers

Answer explanation

Mini-batches allow processing multiple sequences simultaneously, improving training efficiency.

What is the function of the embedding layer in RNNLM?

• It initializes the hidden state
• It compresses output vectors
• It converts word IDs to word vectors
• It reshapes input text to tensors

Answer explanation

The embedding layer transforms word IDs into continuous vector representations for input into the RNN.

What does perplexity measure in language models?

• Speed of training
• Complexity of the model
• The inverse probability of the correct prediction
• Memory usage

Answer explanation

Perplexity is the reciprocal of the predicted word’s probability. Lower perplexity indicates better prediction.

Why is the tanh function used in RNN layers?

• To create a sparse matrix
• To limit output within a fixed range
• To separate classes clearly
• To initialize weights

Answer explanation

The tanh function helps keep the RNN's outputs within a manageable range (-1 to 1), aiding stability.

What does the reset_state() method do in RNNLM?

• It clears the weights
• It deletes the input layer
• It initializes the hidden state
• It stops the model from learning

Answer explanation

reset_state() reinitializes the hidden state in RNNs, ensuring a fresh start for new sequences.