ML B2 CH7

What does an LSTM-based language model generate?

• Probability of word translations
• Probability distribution of the next word
• Fixed-length sentence embeddings
• Correct grammar for sentences

Answer explanation

The LSTM model generates a probability distribution for the next word in a sentence.

What is a deterministic method in sentence generation?

• Randomly choosing any word
• Sampling based on probability
• Choosing the word with the highest probability
• Shuffling the input words

Answer explanation

Deterministic generation selects the word with the highest probability at each step.

What does the seq2seq model consist of?

• Multiple attention layers
• A Transformer and Decoder
• An Encoder and a Decoder
• Two separate RNNs without communication

Answer explanation

Seq2seq models consist of an Encoder that processes input and a Decoder that generates output.

What is the purpose of the encoder in seq2seq?

• Generate translations
• Convert a sequence into a fixed-length vector
• Decode output text
• Create random data

Answer explanation

The encoder transforms a variable-length input sequence into a fixed-length context vector.

What layer is used to convert text into vectors in the encoder?

• Softmax
• ReLU
• Embedding Layer
• Pooling Layer

Answer explanation

The embedding layer maps discrete word indices into continuous vector space representations.

What is the function of the decoder in seq2seq?

• Generate input text
• Interpret vector h and generate output sequence
• Evaluate gradients
• Summarize input only

Answer explanation

The decoder uses the vector h from the encoder to generate the output sequence step by step.

What is the purpose of padding in seq2seq?

• To compress data for memory
• To add noise to training data
• To equalize the length of input and output sequences
• To remove rare words

Answer explanation

Padding ensures all sequences have the same length for batch processing.

How does reversing the input sequence affect seq2seq?

• Decreases performance
• Has no effect
• Improves accuracy
• Increases output size

Answer explanation

Reversing the input helps align important words in short-term memory, improving model accuracy.

What is 'peeky' in seq2seq improvement?

• Adding noise to output data
• Sharing the vector h across multiple decoder layers
• Masking input tokens
• Changing activation functions

Answer explanation

The peeky mechanism improves performance by injecting context (h) into more layers of the decoder.

Which is NOT an application of seq2seq?

• Image classification
• Machine translation
• Chatbots
• Summarization

Answer explanation

Image classification does not require sequence-to-sequence transformation, unlike the other options.