Transformer Quiz

What is the main problem with Recurrent Neural Networks (RNN)?

• Vanishing or exploding gradients

• Easy access to information from long time ago

• Fast computation for long sequences

• Efficient for short sequences

What is the purpose of positional encoding in the Transformer model?

• To capture the position of words in a sentence

• To introduce fluctuations in the data

• To represent the size of the embedding vector

• To relate words to each other

What does Self-Attention allow the model to do?

• Relate words to each other

• Access information from long time ago

• Compute fast for long sequences

• Avoid vanishing or exploding gradients

What is the purpose of layer normalization in the Transformer model?

• To relate words to each other

• To introduce fluctuations in the data

• To make the model causal

• To normalize the output of each layer

What is the goal of Masked Multi-Head Attention in the Transformer model?

• To make the model causal

• To relate words to each other

• To compute fast for long sequences

• To introduce fluctuations in the data

Which task is typically performed by an Encoder Only Transformer?

• Machine translation

• Text summarization

• Question-answering

• Sentimental analysis

What is the primary use case for a Decoder Only Transformer?

• Anomaly detection

• Text classification

• Text generation

• Chatbots

In the Transformer model, what is the purpose of the Beam Search strategy?

• To select the word with the maximum softmax value

• To evaluate all possible next words at each step

• To make the model causal

• To introduce fluctuations in the data

What is the main advantage of using Multi-head Attention in the Transformer model?

• To compute fast for long sequences

• To avoid vanishing or exploding gradients

• To relate words to each other

• To introduce fluctuations in the data

Which type of Transformer model is commonly used for Machine Translation?

• Decoder Only Transformer

• Encoder-decoder Transformer

• Encoder Only Transformer

• Self-Attention Transformer