NEW
Font size
WorksheetsFine-Tuning Quiz Answers
Total questions: 20
Worksheet time: 10mins
What is the primary purpose of fine-tuning a pretrained model?
To adapt models to domain-specific data
To compress model weights for storage
To replace pretraining with faster training
To remove attention layers entirely
In full fine-tuning, which parameters are typically updated?
All parameters of the model
Only embedding matrix weights
Only normalization layer scales
Only output layer parameters
Which scenario most strongly justifies avoiding full fine-tuning due to feasibility?
Model has public checkpoints available
Dataset is very large and proprietary
Evaluation metrics are already standardized
Team has extensive GPU availability
What is a common risk when fully fine-tuning large models on small datasets?
Overfitting and high compute cost
Underfitting and low variance
Improved generalization without cost
Stable training with fewer epochs
What does PEFT stand for in model adaptation?
Parameter-Efficient Fine-Tuning
Parallel-Enabled Forward Training
Probabilistic Encoder Fine-Tuning
Partial-Embedding Feature Transfer
In many PEFT methods, what is the status of the base model during training?
Base model reinitialized
Base model pruned
Base model quantized
Base model frozen
Roughly what percentage of parameters can PEFT reduce training to compared with full fine-tuning?
30–40% of parameters
90–95% of parameters
60–70% of parameters
10–20% of parameters
Which technique introduces low-rank adapters into attention layers to reduce trainable parameters?
Adapters without attention
Prompt Tuning
LoRA
Prefix Tuning
In LoRA, what is inserted inside attention layers to capture updates efficiently?
Low-rank matrices inside attention layers
Sparse pruning masks across all layers
High-rank projections in feedforward blocks
Dropout units replacing value heads
What is a general effect of adding PEFT modules to a model’s architecture?
Replaces layer normalization mechanisms
Increases tokenization vocabulary sizes
Removes positional encodings from inputs
Adds small trainable modules to model layers
When are PEFT techniques particularly advantageous compared to full fine-tuning?
When label noise is extremely high
When latency is the only constraint
When data is unlimited and cheap
When compute resources are limited
Which set of parameters are trained in some minimal PEFT variants like simple adapters?
Only bias terms
Only embedding positions
Only key-value heads
Only normalization gammas
Which sequence best represents a practical fine-tuning workflow for deployment?
Collect → Label → Test-only → Ship
Tokenize → Pretrain → Quantize → Deploy
Load → Choose PEFT → Train → Evaluate → Deploy
Pretrain → Prune → Distill → Deploy
What is a valid strategy to balance adaptation quality and efficiency for large models?
Combining full fine-tuning + PEFT
Using zero-shot without validation
Training only embeddings forever
Skipping hyperparameter search entirely
Which application commonly benefits from fine-tuning for specialized behavior?
GPU memory diagnostics
General file compression
Operating system bootloaders
Domain-specific chatbots
You must adapt a large language model to legal documents with tight GPU budgets. Which approach is most suitable?
Replace attention with convolution layers
Use PEFT to update small adapter modules
Retrain the model from scratch on statutes
Fully fine-tune all layers for many epochs
A team reports that LoRA improved training speed but lost generalization. Which adjustment best addresses this first?
Remove adapters and freeze embeddings
Disable evaluation to save training time
Increase LoRA rank and regularization strength
Lower learning rate while removing decay
During evaluation, a PEFT-tuned model overfits the training set. Which action is most reasonable next?
Freeze more layers and remove adapters
Train longer without checkpoints or metrics
Increase batch size without changing anything
Add validation early stopping and augment data
Which statement best contrasts PEFT with full fine-tuning in deployment contexts?
PEFT stores small deltas, easing versioning
PEFT requires larger checkpoints for shipping
PEFT mandates retraining tokenizers too
PEFT always improves latency regardless
A project needs quick iteration across multiple domains using one base model. Which setup is most effective?
Maintain separate PEFT adapters per domain
Train separate full models for each domain
Use one fixed model without adapters
Replace all attention with static prompts
