Understanding Transformer Models

What is the primary purpose of the Encoder in a Transformer model?

• To generate sequential text outputs

• To process and understand the input data before passing it to the decoder

• To apply attention mechanisms only on the output

• To directly predict the final output

In a Transformer model, what is the key difference between the Encoder and Decoder?

• The Encoder processes input sequences, while the Decoder generates output sequences

• The Encoder uses self-attention, while the Decoder does not

• The Decoder is responsible for processing input sequences, while the Encoder generates outputs

• There is no difference between them

Which of the following architectures is an Encoder-Decoder model?

• BERT

• GPT

• T5

• Word2Vec

How does BERT differ from GPT?

• BERT is bidirectional, while GPT is unidirectional

• GPT is bidirectional, while BERT is unidirectional

• BERT generates text, while GPT is only used for classification

• BERT is trained using autoregressive modeling, while GPT is not

What does the positional encoding in a Transformer do?

• Helps the model understand the order of words in a sequence

• Translates words into numerical vectors

• Removes the need for self-attention

• Reduces computational complexity

What is the purpose of the embedding layer in a Transformer model?

• To convert input words into numerical vectors

• To apply attention mechanisms

• To remove redundant information from input

• To perform sequence classification

In an Encoder-Decoder Transformer model, what is the role of the cross-attention mechanism?

• It allows the decoder to focus on relevant parts of the encoder's output

• It replaces self-attention in the decoder

• It prevents overfitting

• It ensures that the encoder ignores unnecessary information

Which Transformer model is specifically designed for text generation?

• BERT

• GPT

• T5

• ResNet

What loss function is commonly used for training GPT models?

• Mean Squared Error (MSE)

• Cross-Entropy Loss

• Hinge Loss

• Contrastive Loss

Why is self-attention important in Transformer models?

• It allows the model to focus on different parts of the input sequence

• It reduces computational complexity

• It replaces embeddings

• It is used only in the decoder