WorksheetsLinear Algebra and Machine Learning Quiz
Total questions: 111
Worksheet time: 56mins
What does a scalar represent in linear algebra?
A 1D array
A single numerical value
A matrix with one row
A function
Which of the following best describes a matrix?
A list of vectors
A 2D array of numbers
A collection of scalars only
A line in space
If a matrix has dimensions 3×4, it has:
3 columns and 4 rows
4 rows and 3 columns
3 rows and 4 columns
12 columns
In deep learning, vectors and matrices are often used to represent:
Only images
Mathematical constants
Data and model parameters
Programming functions
Which operation is not valid between two matrices?
Addition of same-shaped matrices
Multiplication with proper dimensions
Transposing a matrix
Dividing one matrix by another
The dot product of two vectors results in a:
Matrix
Vector
Scalar
Tensor
What is the transpose of a matrix?
Flipping matrix values
Swapping rows with columns
Taking the inverse
Reducing dimensions
What is required for two matrices A and B to be multiplied (i.e., AB)?
Number of columns in A = number of columns in B
Number of rows in A = number of rows in B
Number of columns in A = number of rows in B
Both must be square matrices
In neural networks, the weight matrices are often:
Used for storing losses
Randomly initialized
Always identity matrices
Deterministically fixed
What is the identity matrix?
A matrix full of zeros
A matrix with all ones
A square matrix with 1s on the diagonal and 0s elsewhere
A matrix with mirrored values
What is classification in machine learning?
Finding clusters in unlabeled data
Mapping input to a discrete output
Finding correlations between variables
Generating continuous outputs
In supervised learning, what is provided with the input data?
Data clusters
Noise
Labels
Hidden layers
Which component defines the possible outputs a learning algorithm can predict?
Hypothesis space
Output space
Feature vector
Cost function
The hypothesis space is:
All possible datasets
All outputs a model can predict
All candidate functions a model can learn
All possible errors a model can make
Which component of learning evaluates prediction error?
Optimizer
Loss function
Gradient
Feature vector
What is the role of the learning algorithm?
To define the loss function
To find the best hypothesis in the hypothesis space
To generate the training data
To normalize inputs
What does the input space represent?
All possible labels
Set of possible inputs to the model
Set of outputs the model can produce
Distribution of the loss function
The feature vector refers to:
The label assigned to a sample
The space of learned weights
A numerical representation of input data
A type of classifier
What is Empirical Risk Minimization (ERM)?
Estimating risk using test data
Calculating the maximum loss
Minimizing loss on training data
Ignoring loss on training data
True risk is also known as:
Empirical error
Test loss
Expected risk
Hypothesis error
What do we aim to minimize in machine learning?
True labels
Training speed
Expected risk
Model size
Generalization is the model's ability to:
Memorize training data
Perform well on training data only
Perform well on unseen data
Store all input-output pairs
Overfitting happens when:
The model generalizes well
The training error is high
The test error is lower than training error
The model memorizes training data
Underfitting occurs when:
The model is too complex
The model fails to learn from training data
The model performs well on test data
The hypothesis space is large
What does the bias in bias-variance tradeoff refer to?
Sensitivity to noise
Error due to incorrect assumptions in the model
Variation due to data sampling
Errors in the test set
Variance in a model refers to:
Noise in the data
Sensitivity to small changes in training data
Assumptions made by the model
Error on the training set
Which of the following can reduce variance?
Increasing model complexity
Decreasing training size
Using simpler models
Using fewer epochs
Which technique can help reduce overfitting?
Increasing training error
Using a large hypothesis space
Regularization
Increasing model depth
High bias and low variance usually lead to:
Overfitting
Underfitting
Perfect generalization
No learning
A model with low bias and high variance tends to:
Underfit
Ignore the data
Overfit
Generalize well
Which of these is a common loss function for classification tasks?
Mean squared error
Cross-entropy
Hinge loss
Both B and C
Cross-validation helps in:
Reducing test error
Estimating model performance on unseen data
Increasing training accuracy
Normalizing input data
What happens if a model is too complex for the data?
It generalizes better
It underfits
It overfits
It becomes regularized
Which component selects the best model based on performance?
Feature extractor
Output function
Learning algorithm
Cost function
The tradeoff between bias and variance is crucial for:
Training speed
Memory efficiency
Model generalization
Choosing optimizers
What was the main topic discussed in the previous week's lecture before introducing deep learning?
Decision Trees
Logistic Regression for Classification
Support Vector Machines
Neural Networks
What framework was used to implement Logistic Regression in the previous week's lecture?
Empirical Risk Minimization (ERM)
Gradient Boosting
K-Means Clustering
Principal Component Analysis (PCA)
What application was built using Softmax Regression in the previous week's lecture?
Image Segmentation
MNIST Handwritten Digit Classification
Speech Recognition
Sentiment Analysis
What limitation of Logistic Regression was observed when dealing with non-linearly separable data?
It cannot handle high-dimensional data
It is only effective for linearly separable data
It requires manual feature selection
It cannot perform multiclass classification
Why is feature extraction a challenge in traditional machine learning for unstructured datasets like images?
It reduces the dataset
It results in high-dimensional data, e.g., 784 columns for 28x28 images
It is computationally inexpensive
It automatically adapts to image variations
What is a key difference between machine learning and deep learning according to the lecture?
Machine learning requires no human intervention
Deep learning can learn features directly from data
Machine learning is only used for image data
Deep learning requires manual feature engineering
Why is manual feature extraction considered a limitation in traditional machine learning?
It is always computationally inexpensive
It requires expertise and may not generalize well
It automatically adapts to different datasets
It eliminates the need for dimensionality reduction
What is a major breakthrough of deep learning mentioned in the lecture?
It has replaced all traditional machine learning algorithms
It has established state-of-the-art results in various pattern recognition tasks
It requires no data preprocessing
It only works with structured datasets
What is a common element in modern deep learning breakthroughs?
Use of decision trees
Neural networks
Manual feature extraction
Linear regression model
What is a common element in modern deep learning breakthroughs?
Use of decision trees
Neural networks
Manual feature extraction
Linear regression models
What is an example application of neural networks mentioned in the lecture?
Auto Caption Generator
K-Means Clustering
Principal Component Analysis
Logistic Regression
What inspired the development of neural networks according to the lecture?
Human ability of thinking
Quantum computing
Statistical modeling
Graph theory
What is the theory of Associationism, as described in the lecture?
Learning is a process of forming associations between temporally related phenomena
Learning is purely based on genetic factors
Learning requires no external stimuli
Learning is a random process
Who proposed the idea that the brain is a network of interconnected neurons in 1873?
Alan Turing
Alexander Bain
Warren McCulloch
Frank Rosenblatt
What is a key difference between connectionist machines and Von Neumann machines?
Connectionist machines use a single processing unit
Connectionist machines store programs in connections between units
Von Neumann machines are non-linear
Connectionist machines are only used for numerical calculations
What are the major elements of a biological neuron mentioned in the lecture?
Dendrite, Synapse, Soma, Axon
Input, Output, Weight, Bias
Node, Edge, Vertex, Graph
Processor, Memory, Bus, Cache
Who proposed the first computational model of a neuron in 1943?
Frank Rosenblatt
Alan Turing
Warren McCulloch and Walter Pitts
Alexander Bain
What type of logic does the McCulloch-Pitts (MCP) neuron model?
Fuzzy logic
Boolean logic
Probabilistic logic
Quantum logic
What happens when an inhibitory synapse is active in an MCP neuron?
It amplifies the excitatory inputs
It prevents the neuron from firing regardless of other inputs
It increases the threshold value
It has no effect on the neuron
What is the output of an MCP neuron when the sum of excitatory inputs is below the threshold?
1
0
-1
Undefined
Which Boolean function cannot be implemented by a single MCP neuron due to its non-linear separability?
AND
OR
XOR
NOT
Who introduced the Perceptron model in 1958?
Warren McCulloch
Frank Rosenblatt
Alan Turing
Marvin Minsky
What is a key feature of the Perceptron compared to the MCP neuron?
It uses fixed weights
It incorporates learnable weights and an adaptive learning process
It only processes binary inputs
It does not use a threshold function
What is the role of the bias term in the Perceptron model?
It scales the input values
It adjusts the decision boundary
It reduces the dimensionality of inputs
It replaces the activation function
Why is the step function unsuitable for real-valued inputs in a Perceptron?
It is non-differentiable and produces binary outputs
It is computationally expensive
It cannot handle binary classification
It requires manual tuning
What is the purpose of the sigmoid activation function in sigmoid neurons?
To introduce non-linearity and map inputs to a range between 0 and 1
To perform linear transformations
To reduce the dimensionality of inputs
To eliminate the need for weights
What is a key limitation of the perceptron discussed in the lecture?
It can handle non-linearly separable data effectively
It only works well for linearly separable data
It uses a sigmoid activation function
It requires no training data
Why is the unit step function not ideal for modern neural network problems?
It is computationally expensive
It is non-differentiable and produces binary outputs
It allows for complex decision boundaries
It supports gradient-based optimization
What problem was highlighted in the tutorial regarding the perceptron's inability to solve certain Boolean functions?
AND function
OR function
XOR function
NOT function
What is a major drawback of the perceptron learning algorithm when applied to multi-layer networks?
It supports backpropagation
It cannot propagate errors to hidden layers
It is computationally efficient for deep networks
It handles non-linear data effectively
What does arranging perceptrons in layers allow, according to the lecture?
Learning linearly separable functions only
Learning non-linearly separable functions like XOR
Reducing the number of neurons required
Eliminating the need for activation functions
What is a key property of a good activation function?
Linear output to simplify computations
Non-differentiability to avoid optimization
Non-linearity to learn complex patterns
Binary output for all inputs
What is the output range of the sigmoid activation function?
-1 to 1
0 to 1
0 to infinity
-infinity to infinity
Why is the sigmoid activation function useful for binary classification?
It produces discrete outputs
It maps inputs to probabilities between 0 and 1
It is non-differentiable
It avoids non-linear decision boundaries
What activation function is typically used in the output layer for multi-class classification?
Sigmoid
ReLU
Softmax
Tanh
What issue can arise from activation functions in deeper networks, as mentioned in the lecture?
Vanishing or exploding gradients
Linear decision boundaries
Overly simple computations
Inability to handle binary inputs
What does the Universal Approximation Theorem state, as per the lecture?
Neural networks cannot represent non-linear functions
Neural networks with non-linear activation functions can represent any non-linear function
Single-layer perceptrons are sufficient for all tasks
Deep networks are less expressive than shallow ones
How many neurons are theoretically required in a single hidden layer to represent any Boolean function with n n n inputs?
n n n
2n 2n 2n
2n 2^n 2n
n2 n^2 n2
Why is a single hidden layer with many neurons impractical for real-world tasks like image recognition?
It is computationally inefficient due to the large number of neurons required
It cannot learn non-linear patterns
It eliminates the need for multiple layers
It is only suitable for binary classification
What defines a shallow neural network, as discussed in the lecture?
A network with no hidden layers
A network with one or a few hidden layers
A network with more than two hidden layers
A network with a single neuron per layer
What is a deep neural network, according to the lecture?
A network with exactly one hidden layer
A network with more than two hidden layers
A network with no activation functions
A network with fewer than 10 neurons
How is the number of neurons in the input layer determined for the MNIST dataset?
It equals the number of classes
It equals the size of the input image (e.g., 784 for 28x28 images)
It is a hyperparameter chosen by the engineer
It depends on the number of hidden layers
What is a hyperparameter in the context of hidden layers?
The number of input neurons
The number of hidden layers and neurons per layer
The activation function used in the output layer
The size of the output layer
What happens if the number of neurons in hidden layers is too high?
The network underfits the data
The network may overfit the data
The network becomes linearly separable
The network requires fewer layers
How many neurons are typically used in the output layer for multi-class classification?
One neuron
Two neurons
Equal to the number of classes
Equal to the number of inputs
What is a fully connected neural network?
A network where only some neurons are connected
A network where all neurons in one layer are connected to all neurons in the next layer
A network with no hidden layers
A netw
What is a requirement for backpropagation, as stated in the lecture?
A non-differentiable activation function
A dataset with input-output pairs
A single-layer network
A fixed learning rate
Which gradient descent variant computes the gradient for the entire training set?
Stochastic Gradient Descent
Mini-Batch Gradient Descent
Batch Gradient Descent
Online Gradient Descent
What is a benefit of Mini-Batch Gradient Descent compared to Batch Gradient Descent?
It uses the entire dataset for each update
It reduces variance in parameter updates for more stable convergence
It is slower but more accurate
It eliminates the need for a learning rate
What is a challenge of gradient descent mentioned in the lecture?
It always converges to the global minimum
It can get trapped in suboptimal local minima or saddle points
It does not require a learning rate
It is only applicable to single-layer networks
What is a key challenge in image classification mentioned in the lecture?
Lack of computational resources
Non-linear decision boundaries for non-linearly separable data
Inability to use activation functions
Absence of labeled datasets
What issue arises when extracting features from images by flattening them into vectors?
Increased computational efficiency
Loss of spatial information between pixels
Improved feature detection
Reduced dataset size
Which of the following is NOT listed as a challenge in processing image data for feature extraction?
Viewpoint variation
Illumination changes
Lack of activation functions
Background clutter
What happens when an image dataset like MNIST is represented as a vector with 784 columns?
The dataset becomes low-dimensional
The dataset becomes high-dimensional, increasing computational complexity
Spatial relationships are preserved
The dataset is automatically classified
What is a consequence of intra-class variation in image classification?
It simplifies feature extraction
It makes it harder to generalize across similar objects
It reduces the need for deep networks
It eliminates the need for pooling layers
What is a strength of FCNs highlighted in the lecture?
They preserve spatial information in images
They use hierarchical layers to learn complex patterns
They require fewer parameters than CNNs
They are designed specifically for image data
Why are FCNs not ideal for image recognition tasks?
They cannot handle non-linear data
They treat pixels as independent features, losing spatial relationships
They are computationally inexpensive
They automatically extract features
What is a major drawback of having too many parameters in an FCN?
Reduced risk of overfitting
Increased computational cost and risk of overfitting
Improved generalization to unseen data
Simplified training process
How many total trainable parameters were calculated for the FCN example with three hidden layers and an output layer?
50,240
94,154
33,024
8,320
What is a proposed solution to reduce the number of parameters in FCNs for image processing?
Increase the number of hidden layers
Extract meaningful features to reduce input dimensionality
Use linear activation functions
Flatten images into larger vectors
What is a key property of a good activation function?
Non-differentiability to simplify optimization
Non-linearity to learn complex patterns
Binary output for all inputs
High computational complexity
Why is the step function not recommended for image classification tasks?
It is computationally expensive
It is non-differentiable, hindering backpropagation
It supports complex decision boundaries
It avoids vanishing gradients
What is a disadvantage of the sigmoid activation function in deep networks?
It is not differentiable
It causes vanishing gradients for large inputs
It is zero-centered
It is computationally inexpensive
What is the primary advantage of the ReLU activation function?
It avoids the vanishing gradient problem for positive inputs
It is zero-centered
It produces binary outputs
It is computationally expensive
How does Leaky ReLU address the "dying ReLU" problem?
It sets all outputs to zero
It allows a small gradient for negative inputs
It eliminates non-linearity
It increases computational complexity
What distinguishes CNNs from FCNs?
CNNs assume inputs are in image format and preserve spatial information
CNNs use only fully connected layers
CNNs are less efficient in parameter usage
CNNs cannot use backpropagation
What is the typical structure of a CNN architecture?
Input → Fully Connected → Output
Input → Convolution → Activation → Pooling → Fully Connected
Input → Pooling → Convolution → Output
Input → Activation → Fully Connected
What is the role of the convolution operation in CNNs?
To flatten the image into a vector
To detect local patterns like edges and textures
To reduce the depth of the feature map
To eliminate non-linearity
What is the purpose of the pooling layer in CNNs?
To increase spatial dimensions
To reduce spatial dimensions while retaining essential information
To introduce learnable parameters
To perform classification
Which pooling operation is commonly used in CNNs?
Sum pooling
Max pooling
Median pooling
Random pooling
How are CNNs trained, according to the lecture?
Using forward pass and backpropagation with gradient descent
Using only forward computation without weight updates
Using manual feature extraction
Without activation functions
What is computed during the backward pass of a convolutional layer?
The input image
Gradients with respect to the filter and input
The final class probabilities
The spatial dimensions of the output
How is the gradient with respect to the filter computed in the backward pass?
As a convolution between the input and the error gradient
As a sum of all pixel values
As a flattening operation
As a pooling operation
What is the role of the fully connected layer in a CNN for image classification?
To extract low-level features
To classify the input based on high-level features
To reduce spatial dimensions
To introduce non-linearity
Why does a deeper CNN with convolutional and pooling layers achieve higher accuracy (91% vs. 86%) compared to a shallow FCN, as mentioned in the lecture?
It uses fewer parameters
It leverages convolution and pooling to extract and preserve spatial features
It avoids backpropagation
It eliminates the need for activation functions
What is a primary challenge in training CNNs mentioned in the lecture?
Inability to use activation functions
Balancing overfitting and underfitting for generalization
Lack of computational resources
Absence of labeled datasets
