NLP-Transformers Last Quiz

What is the main purpose of multi-head self-attention in the Transformer model?

• To ignore certain aspects of the input

• To process inputs sequentially

• To process inputs in parallel

• To learn multiple contextual relationships at once

What is the dimension of each weight matrix in the multi-head self-attention layer if the embedding dimension is 512 and there are 8 heads?

• 512 x 64

• 512 x 128

• 256 x 128

• 256 x 64

What is the purpose of using sinusoids in the original Transformer model?

• To process inputs sequentially

• To reduce the dimension of the input

• To add non-linearity to the system

• To inject position information back into the model

What is the purpose of the pointwise feed forward neural network in the Transformer model?

• To ignore certain aspects of the input

• To process inputs sequentially

• To introduce non-linearity to learn complex relationships

• To reduce the dimension of the input

What is the main training task used in the original BERT model?

• Sentiment Analysis

• Text Generation

• Masked Language Modeling (MLM)

• Next Sentence Prediction (NSP)

What is the purpose of fine-tuning a pre-trained language model?

• To adapt the model for a specific task with less data and time

• To train the model from scratch

• To reduce the dimension of the input

• To ignore certain aspects of the input

What is the main difference between BERT-base and BERT-large models?

• More layers in the decoder

• Larger vocabulary size

• More layers in the encoder

• More blocks, more attention heads, and larger embedding dimensions

If the embedding dimension is 768 and sequence length is 1024 and vocabulary size is 32000 tokens what will be the dimension of softmax MLM head ?

• 1024 x 32000

• 768 x 1024

• 768 x 32000

• depends on the number of self attention heads

What is the purpose of the masked language modeling head in BERT?

• To generate random tokens for training

• To process inputs sequentially

• To predict the correct token for masked words using context

• To reduce the dimension of the input

What is the main advantage of using pre-trained language models like BERT?

• Ignoring linguistic knowledge for specific tasks

• Training the model from scratch for every task

• Reducing the dimension of the input for specific tasks

• Leveraging linguistic knowledge for specific tasks with less training data and time