WorksheetsIntroduction to Artificial Neural Networks
Total questions: 124
Worksheet time: 1hrs 2mins
Which statement best defines an Artificial Neural Network in modern computing?
A computational model inspired by biological neurons
A hardware-only solution for parallel processors
A rule-based expert system using logic trees
A statistical test for comparing two datasets
Why do Artificial Neural Networks occupy a central role in machine learning?
They generalize across vast application areas
They eliminate the need for training data
They only work for small, niche tasks
They replace all other AI techniques entirely
In the context of ANNs, what does Explicit Learning most likely refer to?
Clear, structured learning with defined targets
Unsupervised discovery without labeled data
Random exploration through trial-and-error
Implicit pattern formation without guidance
Which model by McCulloch and Pitts in 1943 provided a mathematical basis for early neural networks?
Threshold Logic Model
Linear Regression Model
Bayesian Inference Model
Hidden Markov Model
Hebbian Learning is best characterized as which learning paradigm?
Supervised learning rules
Semi‑supervised learning rules
Reinforcement learning rules
Unsupervised learning rules
Who is credited with inventing the perceptron in 1958?
Paul Werbos
Donald Hebb
Warren McCulloch
Frank Rosenblatt
What core operation did early perceptrons learn across two layers?
Addition and subtraction
Convolution and pooling
Multiplication and division
Exponentiation and log
Which researcher formulated back‑propagation as a learning algorithm in 1975?
Seymour Papert
Paul Werbos
Alan Turing
Marvin Minsky
Research in neural networks declined between 1960 and 1975 largely due to which paired issues?
Limited data and poor sensors
Weak theories and ethical bans
Costly hardware and few researchers
Simple models and slow computers
Which development in the 1980s–1990s most enabled the resurgence of ANN research?
Quantum processors widespread
Analog circuit miniaturization
Parallel computing processors growth
Optical networking standardization
Farley and Clark’s work in 1954 used computation machines and simulations at which institution?
Stanford University
Carnegie Mellon University
Harvard University
Massachusetts Institute of Technology
In the biological neuron diagram, which structure primarily receives signals from other neurons?
Axon extending from the cell body
Nucleus located within the soma
Synapse terminals at the axon tip
Dendrites branching near the soma
Which pairing best maps a biological component to its artificial neuron counterpart?
Dendrites to input features x
Nucleus to adjustable weights w
Axon to activation function g
Synapse to network topology
In the artificial neuron diagram, what is computed immediately before the activation function block?
Bias subtraction from output
Weighted sum Σ of inputs
Gradient of loss function
Binary thresholding decision
Which statement correctly describes weights in an artificial neuron?
Scalars that scale each input
Outputs that store final predictions
Functions that compute gradients
Hyperparameters that define topology
Which set lists valid input types for an artificial neuron?
Continuous, discrete, categorical
Nominal, ordinal, ratio
Text, image, audio
Integer, real, binary
What role does the activation function play in an artificial neuron?
Normalizes inputs before summation
Routes signals between layers
Transforms summed input to output
Stores weights during training
Which description best matches ANN topology?
Range of valid input formats
Magnitude of each weight parameter
Type of activation function used
Arrangement of neurons and connections
Which activation function outputs exactly two levels, switching at x = 0 as shown in the diagram?
Symmetric linear function
Binary hard-limit function
Bipolar hard-limit function
Continuous sigmoid function
In the plotted symmetric activation, f(x) equals x. What property does this function exhibit across positive and negative inputs?
Saturating at large magnitudes
Discrete step transitions
Even symmetry around y-axis
Odd symmetry around origin
Which curve approaches 0 and 1 asymptotically and is given by f(x)=1+e−x1 ?
Binary step function
Logistic sigmoid function
Piecewise stairs function
Hyperbolic tangent function
Which activation maps negative inputs to −1 and positive inputs to +1 as depicted?
Sigmoid logistic function
Linear symmetric function
Binary hard-limit function
Bipolar hard-limit function
The tanh activation shown has formula (ex−e−x)/(ex+e−x) . What is its output range?
From −1 to +1
From 0 to +1
From −∞ to +∞
From 0 to +2
Which activation illustrates multiple discrete plateaus labeled a, b, c with outputs 0,1,2,3?
Symmetric linear function
Continuous tanh function
Multi-level steps function
Binary threshold function
Which ANN topology connects inputs directly to outputs without hidden layers, as drawn?
Single-layer neural network
Feedback single-layer network
Multi-layer neural network
Self-organizing network
Which topology shows recurrent connections from outputs back into the network, enabling feedback?
Purely linear perceptron
Feedback single-layer network
Self-organizing network
Multi-layer feedforward network
Which ANN property enables multiple processing units to work simultaneously to increase speed?
Parallelism and synchronization
Distributed memory storage
Generalization capability
Ease of construction
What does adaptation in ANNs primarily allow the network to do?
Store data permanently
Ignore input variations
Adjust to problem-specific data
Follow fixed program steps
Distributed memory in ANNs means information is stored how?
Only during training sessions
Externally in databases
Across many processing elements
In one central unit
Generalization in ANNs refers to the ability to do what after training?
Memorize training samples
Perform on unseen but related data
Require retraining for each case
Eliminate all input noise
Fault tolerance in ANNs helps primarily with which situation?
Precise feature engineering
Long-term storage capacity
Errors in inputs or processing
Sequential execution speed
Problem definition in ANNs is important because the network learns from what?
Purely manual tuning
Prewritten program rules
A set of modeled inputs
Random unstructured inputs
Handling noisy data in ANNs typically involves which capability?
Reject all imperfect samples
Disable learning on noise
Infer patterns despite errors
Require complete datasets
Ease of construction and learning in ANNs is often due to what?
Highly customized code
Standard algorithms and rules
Manual configuration only
Hardware-only solutions
Parallelism in ANNs contrasts most directly with which architecture style?
Von Neumann sequential processing
Event-driven reactive systems
Quantum probabilistic computing
Pipeline vector processing
Which property helps ANNs avoid overreliance on exact training samples?
Generalization capability
Fault tolerance
Distributed memory
Adaptation ability
If some feature values are missing, which ANN property still enables useful outputs?
Fault tolerance
Problem definition
Parallelism
Ease of construction
Why is clear problem definition critical before training an ANN?
It removes need for data collection
It allows fixed rule execution
It sets algorithmic programming steps
It determines network architecture
Adaptation in ANNs reduces dependence on what traditional approach?
Rule-based programming
Gradient descent updates
Data augmentation schemes
Batch normalization layers
Distributed memory contributes to robustness because information is what?
Redundant across units
Compressed in one node
Encrypted after training
Discarded when noisy
Generalization is commonly assessed by performance on which dataset?
Training set only
Validation or test set
Randomized noise samples
Memory recall set
Fault tolerance means the network can do what when inputs contain errors?
Require manual fixes
Retrain from scratch
Correct or compensate
Stop processing entirely
Which pairing best matches property to benefit: Parallelism → ?
Better storage through redundancy
Higher speed through concurrency
Simpler coding via templates
Improved accuracy via rules
Which scenario best illustrates adaptation ability?
Data is stored in one memory
Inputs are strictly noise-free
Weights update to fit task data
A fixed program runs identical steps
Which scenario best demonstrates handling noisy data?
Model discards all corrupted rows
Model stops on any outlier
Model infers despite missing values
Model requires perfect labels
Ease of construction in ANNs often results from availability of what?
Hand-written case rules
Standardized learning procedures
Manual feature extraction
Custom hardware only
Which statement best describes generalization in artificial neural networks?
Memorizing all training examples exactly
Optimizing weights without any data
Stopping training when loss reaches zero
Learning with minimal training data size
According to the diagram, how does ANN performance typically change as the learning data size increases?
Gradually increases over larger data
Rapidly decreases with more data
Remains constant regardless of data
Oscillates unpredictably with size
In supervised learning, what indicates that the learning process should stop?
Bias value equals learning rate
Clusters fully formed without labels
Weights become stable on inputs
Network learns input–output mappings
Which learning type relies solely on input data without target outputs?
Unsupervised learning approach
Supervised learning approach
Reinforcement learning approach
Self-organization approach
Self-organization in ANNs is most similar to which learning type?
Unsupervised learning methods
Semi-supervised methods
Active learning methods
Supervised learning methods
Which technique is commonly associated with self-organization processes?
Clustering of input patterns
Gradient descent on loss
Labeling with target outputs
Backpropagation with labels
Which learning rule updates weights proportionally to the product of input and output?
Hebb rule formulation
Widrow–Hoff formulation
Competitive rule formulation
Generalized delta formulation
The Widrow–Hoff rule adjusts weights using which error term?
Difference between target and output
Difference between bias and weight
Sum of input and output values
Product of input and output values
In the competitive rule, the update depends on which comparison?
Input minus current weight
Target minus current output
Bias minus learning rate
Momentum minus decay factor
The Generalized Delta Rule uses which quantity to scale the input during weight updates?
Fixed bias threshold
Output normalization value
Delta of the neuron j
Random momentum term
Adding momentum to the GDR introduces which extra term in the update?
Divide by eta learning rate
Add beta times bias
Subtract alpha times weight
Multiply gamma times input
What does the learning rate (η) control during training?
Number of hidden neurons
Magnitude of bias values
Choice of activation function
Step size of weight changes
In ANN models, what role does the bias or threshold (θ) primarily play?
Replaces the learning rate
Shifts neuron activation level
Sets training data size
Determines output labels
Who is credited with developing the back-propagation algorithm and in which year was it introduced?
Paul Werbos in 1975
Geoff Hinton in 1986
Yann LeCun in 1998
Alan Turing in 1950
Which learning paradigm best describes training in a Back-Propagation Neural Network?
Unsupervised learning
Reinforcement learning
Supervised learning
Self-supervised learning
What is the primary role of non-linear activation functions in multi-layer networks?
Reduce network connectivity
Enable linear separability
Allow complex function approximation
Fix weight initialization issues
In the forward-backward process of BP-NN, what happens during the backward phase?
Inputs propagate to outputs
Errors propagate to update weights
Bias terms are removed entirely
Weights are frozen for stability
Which topology characteristic is depicted for BP-NN in the diagram?
Single-layer perceptron
Sparse connections only
Recurrent feedback loops
Full connection between layers
What does the learning rate η control during training?
Magnitude of weight updates
Number of hidden layers
Size of training dataset
Choice of activation function
Which statement best describes the momentum term α in gradient-based updates?
Adds random noise to gradients
Accumulates past updates to speed convergence
Sets an absolute threshold for activation
Fixes weights to prevent overfitting
In a BP-NN with A inputs, B hidden nodes, and C outputs, what is the correct shape of the first-layer weight matrix W1?
A×B matrix mapping inputs to hidden
C×B matrix mapping outputs to hidden
B×A matrix mapping hidden to inputs
A×C matrix mapping inputs to outputs
Which parameter controls the magnitude of weight updates during training in back-propagation?
Activation function
Batch size P
Learning rate η
Output count C
Hidden node net input v is typically computed as which operation?
Cumulative product of inputs
Softmax over inputs X
Element-wise product X⊙W1
Weighted sum W1·X
For a sigmoid unit, which formula gives the hidden node activation Hj from net input v?
Hj = tanh(v)
Hj = 1+e−v1
Hj = ReLU(v)
Hj=v2
Given hidden activations H and second-layer weights W2, how is the output node net input computed?
Difference between first two hidden nodes
Max pooling of hidden nodes
Average of hidden activations
Weighted sum over hidden nodes
Which expression represents the error across C outputs for one sample when using root-sum-squared error?
error = sqrt(Σk(Yk−Ok)2)
error = Σ_k (Yk − Ok)
error = mean(Ok)
error = ΣkOk2
What is the output-layer delta term δ2_k for a sigmoid output unit?
Ok(1−Ok)(Yk−Ok)
Ok+Yk
Ok(1+Ok)(Yk+Ok)
Ok−Yk
Which term propagates error to the hidden layer in back-propagation?
P = number of samples
η = Σ_i Xi
Zk = Ok − Yk
δ1_k = Zk(1−Zk) Σ_j δ2_j W2_{k,j}
During weight update for W2, which formula is used for a single sample?
W2_{i,j} ← W2_{i,j} − η Zk
W2_{i,j} ← W2_{i,j} + η Ok
W2_{i,j} ← W2_{i,j} − η Xi
W2_{i,j} ← W2_{i,j} + η δ2_j H_i
During weight update for W1, which quantity multiplies the input Xi?
Error scalar only
Output-layer delta δ2_k
Output value Ok
Hidden-layer delta δ1_j
What is the correct iterative structure for training with back-propagation?
Randomly initialize each epoch only
Update weights before computing outputs
While error not acceptable, loop over samples
Compute outputs once, then end
Which step occurs first in the BP-NN algorithm?
Calculate hidden-layer deltas
Compute output node values
Initialize parameters and activation
Save the last weight values
What does random initialization of W1 and W2 primarily prevent?
Symmetry among neurons
Need for activation functions
Exact gradient computation
Overfitting immediately
If the error is acceptable, what should the algorithm do next?
End the while loop
Increase the learning rate
Reset the weights
Recompute deltas repeatedly
Which part of the algorithm saves the trained model state?
Save the last weight values
Calculate δ2 at output
Initialize parameter A
Compute hidden activations
In this worked example, which activation function is used for hidden and output nodes when computing h1, h2, o1, and o2?
Hyperbolic tangent function
Softmax normalization
Logistic sigmoid function
Rectified linear unit
Given input sample [1 0 0] and W1 weights shown, what is the computed value of h1 using the activation applied to sum1 = 0.2?
h1 ≈ 0.4502 after activation
h1 ≈ 0.7311 after activation
h1 ≈ 0.5498 after activation
h1 ≈ 0.5915 after activation
For the same sample, Sum2 at the output layer is 0.805 from hidden activations and W2. After activation, what is o2 approximately?
o2 ≈ 0.551 using activation
o2 ≈ 0.37 using activation
o2 ≈ 0.691 using activation
o2 ≈ 0.525 using activation
The desired output is [1 0], while the actual output is [0.5915 0.691]. Which statement best describes the error acceptance decision shown?
The error is acceptable and training stops
The error is not acceptance so continue training
The error equals zero so weights remain
The error is averaged then discarded
Using δ values at the output layer, which expression matches δ2 for o1 given o1 = 0.5915?
δ2 = o1(1−o1)(1−desired1)
δ2 = o1(1−o1)(desired1−o1)
δ2 = o1(1−o1)(1−o1)
δ2 = o1(1−o1)(o1−desired1)
Which formula correctly computes hidden-layer error δ1₁ using h1, (1−h1), and weighted sum of output deltas through W2?
δ1₁ = h1(1−h1)(W2₁₁δ2₁ + W2₂₁δ2₂)
δ1₁ = h1(1−h1)(W2₁₁ + W2₂₁)
δ1₁ = h1(1−h1)(δ2₁ + δ2₂)
δ1₁ = h1(1−h1)(W1₁₁δ2₁ + W1₂₁δ2₂)
After applying learning rate η = 0.6, which update rule is used for W2 elements, for example W2₁,₁?
W2new = W2old − η·δ2·x
W2new = W2old + η·δ2·x
W2new = W2old − η·δ1·h
W2new = W2old + η·δ2·h
Which challenge most directly concerns selecting appropriate learning rate and momentum values in back‑propagation networks?
Selecting a weight update formalism
Choosing suitable non‑linear activation function
Computing error ratio sample‑wise or cumulatively
Difficulty determining parameter values reliably
Deciding hidden layer count and node quantity
What is a typical approach for deciding the reasonable number of hidden nodes in a back‑propagation model when no canonical rule exists?
Conduct targeted experiments iteratively
Use fixed formula from theory
Always match input feature count
Choose maximum nodes possible
Rely on default library settings
Why does the hidden layer size strongly influence convergence in back‑propagation networks?
It fixes learning rate automatically
It guarantees global minima found
It eliminates activation choice
It removes need for bias nodes
It alters weight search complexity
When selecting activation functions for back‑propagation, what is the key consideration stated?
Avoid any non‑linearities
Select based on output size
Use sigmoid exclusively
Prefer linear functions always
Choose one or more carefully
Which statement best captures the issue around weight update approaches in back‑propagation?
Multiple formalisms exist; pick carefully
There is one standard method always
Updates never affect convergence
Weights only change at initialization
Momentum is unnecessary and harmful
What uncertainty surrounds bias nodes in back‑propagation networks?
Whether they store training data
Whether they must be negative
How many to include per layer
If they replace hidden neurons
Why they prevent overfitting
Which methods are contrasted for error ratio computation in back‑propagation training?
Probabilistic versus deterministic
Sample‑by‑sample versus accumulative
Static versus dynamic typing
Supervised versus unsupervised
Analytical versus symbolic math
A practitioner faces no canonical guidance on learning rate η. What is the recommended path forward?
Freeze η at 0.1 for all tasks
Derive η from bias counts
Run experiments to choose values
Use α to replace η entirely
Ignore η and train anyway
In a network with multiple hidden layers, what complication is emphasized?
Eliminating the output layer entirely
Fixing weights without training
Needing only one activation function
Determining nodes per hidden layer
Removing biases from all layers
What pairing of parameters is explicitly cited as difficult to set in back‑propagation?
Kernel size and stride
Input scaling and centering
Dropout rate and seed
Batch size and epochs
Learning rate and momentum
Which choice reflects a realistic misconception about activation functions to test understanding?
Hidden node count determines function
Bias nodes decide the function used
Linear functions suffice for all tasks
Non‑linear options must be considered
Error ratio method selects function
When comparing error computation strategies during training, what must be decided?
Use per‑sample or accumulate errors
Fix hidden nodes to equal inputs
Avoid biases in all layers
Select linear activation always
Prefer momentum over learning rate
Who introduced the neural network known as the Kohonen network?
Geoffrey Hinton
John Hopfield
Teuvo Kohonen
Yann LeCun
Which learning paradigm is primarily used in Kohonen neural networks?
Reinforcement learning
Unsupervised learning
Semi-supervised learning
Supervised learning
What principle drives weight adaptation in Kohonen networks?
Competitive learning
Hebbian supervision
Backpropagation gradients
Bayesian inference
In a typical Kohonen network architecture, how many layers are emphasized?
One processing layer
Two main layers
Three deep layers
Four hierarchical layers
Which metric commonly determines the winning unit for an input in a Kohonen map?
Cosine similarity
Hamming distance
Euclidean distance
Manhattan distance
What characterizes the neighborhood function in a Kohonen map?
Ignores spatial arrangement
Updates only the winner
Updates distant neurons strongly
Updates nearby neurons together
Which application aligns best with Kohonen networks?
Sequence prediction
Image segmentation
Clustering of high-dimensional data
Exact function approximation
What is a typical topology for a Kohonen neural network?
Fully connected recurrent ring
Two-dimensional lattice map
Tree-structured decision nodes
Feedforward multilayer perceptron
In a Kohonen neural network, what does D(j) represent when comparing a sample to cluster j?
Sum of squared feature differences
Dot product between sample and weights
Learning rate scaled distance
Euclidean norm of weight vector
When selecting the winning cluster for a sample in Kohonen learning, which criterion is used?
Minimum D(j) across clusters
Maximum D(j) across clusters
Largest learning rate alpha
Highest input node count
Which step occurs immediately after computing D(j) for all clusters during one sample update?
Increase the sample index
Decrease the number of clusters
Select the minimum distance
Regenerate the weight matrix
What is the standard weight update for the winning cluster column j in Kohonen learning?
w_i,j = w_i,j − αX_i
w_i,j = αX_i + w_i,j
w_i,j = w_i,j + α(X_i − w_i,j)
w_i,j = w_i,j / α
During initialization of a Kohonen network, which parameters are set before training begins?
Kernel size, stride, padding scheme
Input nodes N, clusters M, samples P, α
Bias terms, dropout rate, momentum
Hidden layers, activation type, batch size
Which linear update rule for the learning rate α is valid with β > 0?
α_new = α − β
α_new = α × β
α_new = α + β
α_new = log(α)
Which non-linear alpha update uses a multiplicative decay with 0 < β < 1?
α_new = α − β
α_new = α + β
α_new = α / β
α_new = α β
In the non-linear schemes, which choice increases α when β > 1?
α_new = α − β
α_new = log(α)
α_new = α β
α_new = α / β
In the example, how many clusters are used in the Kohonen network?
Two clusters are defined
Three clusters are defined
Four clusters are defined
One cluster is defined
Which formula is used to compute D(j) for a sample in the worked example?
Sum of squared input–weight differences
Dot product of input and weights
Sum of absolute input–weight differences
Product of inputs and weights
For the first sample 1 1 0 0 0, which cluster wins based on the minimum D(j)?
Cluster j = 1 wins
Cluster j = 2 wins
Both clusters tie
No cluster is selected
After the first sample update with α = 0.6, what is the new weight w1,1?
0.72 after update
0.70 after update
0.68 after update
0.76 after update
For the sample 0 0 0 1 1, which cluster has the smaller distance D(j)?
Decision cannot be determined
Cluster j = 1 has smaller distance
Cluster j = 2 has smaller distance
Distances are equal
When α is updated by α := α − β with α = 0.6 and β = 0.1, what is the new learning rate?
0.6 after reduction
0.7 after reduction
0.4 after reduction
0.5 after reduction
Using α = 0.5, for the first sample re-run, which cluster wins given d1 = 0.9235 and d2 = 2.6544?
Both clusters win equally
No winner due to tie
Cluster 2 wins with smaller d2
Cluster 1 wins with smaller d1
Across iterations, what operation is repeatedly applied to the winning cluster’s weights?
Randomly reset weights to zero
Move weights away from the input vector
Move weights toward the input vector
Swap weights between clusters
