NEW
Font size
WorksheetsPerceptrons and Activation Functions
Total questions: 15
Worksheet time: 8mins
What is the primary role of a non-linear activation function in a perceptron?
To speed up the training process
To decrease the number of parameters in the network
To reduce the computational complexity of the network
To introduce non-linearities that allow the network to learn complex, non-linear relationships
Which of the following is NOT a common activation function used in deep neural networks?
Square function
Rectified Linear Unit (ReLU)
Hyperbolic tangent (tanh)
Sigmoid function
In a perceptron, the output ŷ is computed as ŷ = g(w0 + Σ(xi·Wi)). What does w0 represent?
Bias term
Output weight
Activation function parameter
Input weight
Why are fully connected layers in neural networks called "Dense" layers?
Because they compress the information densely
Because they have high density of parameters
Because all inputs are densely connected to all outputs
Because they use dense matrix operations
In binary classification with probabilistic outputs, which loss is most appropriate to minimize?
Hinge loss for margin maximization
Categorical cross-entropy with many classes
Mean squared error on probabilities
Binary cross-entropy over predicted probabilities
Which statement best describes the role of the learning rate η in gradient descent?
It scales the gradient to set update step size
It measures validation accuracy each epoch
It normalizes inputs to the network layers
It counts the number of training epochs
Backpropagation computes which quantity at each layer of a neural network?
Forward activations for new inputs
Regularization penalties only
Gradients of loss via the chain rule
Confusion matrix for predictions
Compared with full-batch gradient descent, mini-batch SGD primarily offers which benefit?
Guaranteed monotonic loss decrease
Exact gradient using the entire dataset
No need to tune the learning rate ever
Faster, noisy updates using subsets of data
What is the principal effect of dropout during training?
Randomly zeroes activations to limit co-adaptation
Replaces backpropagation with random search
Permanently deletes neurons from a trained model
Reduces model depth by merging adjacent layers
Early stopping helps prevent overfitting by doing what when training proceeds?
Halting when validation loss begins to rise
Stopping immediately after training loss hits zero
Continuing until training and validation losses match
Halting after a fixed number of parameter updates
In a convolution operation, what does the filter (kernel) do?
Reduces the image size
Computes element-wise multiplication of weights with patches and sums outputs to detect features
Normalizes pixel values
Applies non-linearity
What are the two main benefits of pooling layers in CNNs?
Increasing model complexity and improving training speed
Reducing dimensionality and providing spatial invariance
Improving accuracy and increasing memory usage
Replacing activation functions and reducing bias
In a typical CNN for classification, what is the correct sequence of operations?
Flatten → Fully Connected → Convolution → Pooling → Softmax
ReLU → Convolution → Pooling → Softmax → Fully Connected
Convolution → ReLU → Pooling → Convolution → ReLU → Pooling → Flatten → Fully Connected → Softmax
Convolution → Pooling → ReLU → Flatten → Fully Connected → Softmax
As you progress through deeper layers in a CNN, what type of features are typically learned?
Only low-level features like edges
A hierarchy from low-level features (edges) to mid-level features (parts) to high-level features (objects)
Random features that are not meaningful
Identical features at all depths
Why do convolutional layers scale better than fully connected layers for images?
They eliminate the need for activation functions
They operate faster by ignoring spatial relationships
They exploit local connectivity and weight sharing to reduce parameters while capturing spatial structure
They work only with color images
