WorksheetsNLP-Transformers Last Quiz
Total questions: 10
Worksheet time: 5mins
What is the main purpose of multi-head self-attention in the Transformer model?
To ignore certain aspects of the input
To process inputs sequentially
To process inputs in parallel
To learn multiple contextual relationships at once
What is the dimension of each weight matrix in the multi-head self-attention layer if the embedding dimension is 512 and there are 8 heads?
512 x 64
512 x 128
256 x 128
256 x 64
What is the purpose of using sinusoids in the original Transformer model?
To process inputs sequentially
To reduce the dimension of the input
To add non-linearity to the system
To inject position information back into the model
What is the purpose of the pointwise feed forward neural network in the Transformer model?
To ignore certain aspects of the input
To process inputs sequentially
To introduce non-linearity to learn complex relationships
To reduce the dimension of the input
What is the main training task used in the original BERT model?
Sentiment Analysis
Text Generation
Masked Language Modeling (MLM)
Next Sentence Prediction (NSP)
What is the purpose of fine-tuning a pre-trained language model?
To adapt the model for a specific task with less data and time
To train the model from scratch
To reduce the dimension of the input
To ignore certain aspects of the input
What is the main difference between BERT-base and BERT-large models?
More layers in the decoder
Larger vocabulary size
More layers in the encoder
More blocks, more attention heads, and larger embedding dimensions
If the embedding dimension is 768 and sequence length is 1024 and vocabulary size is 32000 tokens what will be the dimension of softmax MLM head ?
1024 x 32000
768 x 1024
768 x 32000
depends on the number of self attention heads
What is the purpose of the masked language modeling head in BERT?
To generate random tokens for training
To process inputs sequentially
To predict the correct token for masked words using context
To reduce the dimension of the input
What is the main advantage of using pre-trained language models like BERT?
Ignoring linguistic knowledge for specific tasks
Training the model from scratch for every task
Reducing the dimension of the input for specific tasks
Leveraging linguistic knowledge for specific tasks with less training data and time
