AI Advanced 3

In supervised learning, the assumption that training and future data come from the same distribution is critical. Why is this assumption important?

• It ensures the model never needs generalization

• It allows the model to ignore training data entirely

• It guarantees that overfitting improves performance

• If the future data distribution matches the training data distribution, learned patterns are likely to generalize effectively

Overfitting in a supervised learning model occurs when:

• The model’s complexity is lower than necessary

• The model generalizes well to unseen data

• The model ignores training data patterns

• The model fits noise and peculiarities of the training set too closely, harming its performance on new data

Decision trees are prone to overfitting. How do techniques like pruning address this issue?

• By making the tree more complex

• By ignoring data attributes entirely

• By preventing any splits from being made

• By removing branches that do not significantly improve predictive accuracy on validation data, thus improving generalization

Precision and recall are favored metrics over accuracy in certain situations. In which scenario is this especially true?

• When class distributions are even and all errors have equal cost

• When no data is labeled

• When accuracy alone reflects all necessary performance aspects

• When dealing with imbalanced classes or when certain error types are more critical, making a single accuracy value insufficient

Cross-validation provides a more reliable estimate of a model’s performance than a single holdout set. Why?

• It uses the same single split repeatedly

• It relies on no test data

• It ensures that training and testing sets never vary

• It averages performance across multiple folds, reducing the influence of any particular data split’s peculiarities

Support Vector Machines (SVMs) rely on maximizing margins between classes. In complex datasets where linear separability is not possible, how do SVMs adapt?

• They fail to classify the data correctly

• They remove all complex features

• They rely solely on linear kernels

• They use kernel functions to project data into higher-dimensional feature spaces, enabling nonlinear decision boundaries

k-Nearest Neighbor (kNN) is a lazy learner. What is the main trade-off of its simplicity?

• kNN requires complicated training but makes classification instant

• kNN never achieves high accuracy

• kNN cannot handle multiple classes

• kNN is simple and requires no training time, but classification can be slow because it searches the entire training set at query time

Ensemble methods like boosting combine multiple weak learners. Why does boosting often improve performance over a single learner?

• By ignoring the data distribution entirely

• By training the same weak learner repeatedly without adjustments

• By ensuring each weak learner makes random decisions

• By sequentially focusing on misclassified examples and iteratively refining the ensemble, it reduces bias and variance, producing a stronger model

In supervised learning, what is a primary reason to use cross-validation rather than a single holdout test set?

• A single holdout set is always sufficient

• Cross-validation provides a more stable and reliable estimate of generalization performance by mitigating variance due to a single split

• Cross-validation reduces computation since it runs fewer experiments

• Cross-validation does not improve reliability