NEW
Font size
WorksheetsMid-Level MCQs on Transformers and LLMs (Part 1)
Total questions: 25
Worksheet time: 13mins
Which component of the Transformer is responsible for capturing relationships between tokens regardless of distance?
Attention mechanism
Positional encoding
Feed-forward network
What is the main advantage of self-attention in Transformers?
It reduces training data requirements
It processes all tokens in parallel
It uses fewer parameters than RNNs
Which of the following best defines an embedding?
A compressed text file
A numerical vector representing semantic meaning
A special token representing sentence boundaries
Vector databases store embeddings primarily to support:
GPU acceleration
Fast semantic search
Long-term backup
In Retrieval-Augmented Generation (RAG), the retriever’s main role is to:
Generate final answers
Convert prompts to embeddings
Fetch relevant documents
Which training step adjusts model parameters to minimize loss?
Forward pass
Tokenization
Backward pass
What does “fine-tuning” typically modify in a pretrained model?
Tokenizer
Internal weights
GPU configuration
Which method is used to represent token sequences numerically before entering a Transformer?
Positional indexing
Word clouds
One-hot encoding
In a multi-agent LLM system, agents primarily interact through:
Static JSON files
GPU kernel calls
Shared memory and messages
Which loss function is commonly used for language modeling?
Cross-entropy loss
Mean squared error
Hinge loss
Function calling in LLMs mainly helps with:
Reducing GPU load
Improving tokenization speed
Structured and deterministic outputs
Which similarity metric is most commonly used in vector databases?
Manhattan distance
Cosine similarity
Hamming distance
The key difference between RNNs and Transformers is that Transformers:
Work only on images
Use recurrence for sequence modeling
Use attention instead of recurrence
What is the main benefit of batching during model training?
Increases dataset size
Improves computational efficiency
Reduces model parameters
Which of the following is NOT a common use of embeddings?
Increasing batch size
Recommendation systems
Semantic search
In RAG, a reranker is used to:
Generate final summaries
Reorder retrieved results based on relevance
Clean up documents before indexing
Which component of the Transformer helps in capturing nonlinear relationships?
Layer normalization
Feed-forward layers
Softmax operation
What is the output of the softmax layer in classification tasks?
Embedding vectors
Normalized probabilities
Raw logits
During fine-tuning, overfitting can be reduced using:
More parameters
Larger learning rate
Dropout
What does the attention score represent?
The gradient magnitude during backprop
Relevance of one token to another
How likely a token is to be masked
Which indexing method is commonly used in vector databases for fast similarity search?
KD-tree
Bloom filter
HNSW graph
Hash maps
In model evaluation, what does perplexity primarily measure?
Model size in parameters
Training dataset diversity
GPU memory consumption
How well the model predicts text
In a multi-agent LLM system, what is the planner agent mainly responsible for?
Serving the user interface layer
Generating embeddings for documents
Decomposing tasks into subgoals
Translating languages between agents
Which tokenization method is used in most modern large language models?
Character splitting only
Whitespace segmentation
Byte-Pair Encoding (BPE)
Rule-based tokenization
What is the primary purpose of positional encoding in transformer models?
Improve gradient stability in layers
Inject order information into tokens
Encode overall sentence length
Reduce hallucinations during decoding
