Font size
WorksheetsMIS 447 ML - Quiz 2
Total questions: 21
Worksheet time: 34mins
The primary focus of Bagging ensemble methods, such as Random Forest, is to reduce which type of error?
Redundancy
Variance
Noise
Bias
In an ensemble learning context, which statement accurately describes the training process difference between Bagging and Boosting?
Bagging combines predictions using a weighted average, while Boosting uses majority voting.
Bagging models are trained sequentially, while Boosting models are trained in parallel.
Bagging models are trained in parallel, while Boosting models are trained in series (sequentially).
Bagging trains base models independently and in parallel, while Boosting trains models in series, where each model depends on the previous one's errors.
The Random Feature Selection step in the Random Forest algorithm is critical because it ensures:
Optimal hyperparameters are selected for each tree automatically.
The calculation of errors (residuals) for the next tree is accurate.
The majority voting process is always computationally linear.
Individual trees are decorrelated, which reduces the overall ensemble's variance and prevents overfitting.
How does a Random Forest model determine its final prediction for a new data point in a classification task?
By using a weighted average based on individual tree performance.
By using only the prediction from the single best-performing tree.
By calculating the simple average of all individual tree predictions.
By selecting the class that receives the most 'votes' (majority voting) from the trees.
The fundamental idea behind the AdaBoost (Adaptive Boosting) algorithm is to:
Adjust the weight of misclassified samples, forcing subsequent weak learners to focus on these 'harder' data points.
Sequentially train models to predict the residual errors made by the previous ensemble.
Use only deep, unpruned Decision Trees to capture complex patterns.
Train multiple independent weak models on different random subsets of the data in parallel.
Which of the following is a primary disadvantage of the AdaBoost algorithm compared to other ensemble methods?
It is not easily parallelizable due to its sequential nature.
It generally results in low predictive accuracy.
It must use complex, deep learners as base models.
It is highly sensitive to noisy data or outliers, which can be assigned very high weights.
Unlike AdaBoost which re-weights samples, Gradient Boosting (GBM) models achieve sequential improvement by training each new model to predict what?
The residuals (errors) made by the combined predictions of the previous models.
The feature importance scores of the initial model.
The optimal learning rate for the entire ensemble.
The bootstrap samples from the original dataset.
A major drawback of traditional Gradient Boosting Machine (GBM) models compared to Random Forest is:
They suffer from low predictive accuracy.
Their core objective is to increase bias.
The inability to handle categorical features.
The computational intensity and slower training speed due to their sequential nature.
What is a key algorithmic enhancement that makes XGBoost "eXtreme" and helps prevent overfitting, differentiating it from traditional GBM?
It exclusively uses Decision Stumps as base learners.
It incorporates L1 (Lasso) and L2 (Ridge) regularization into the objective function.
It always uses majority voting for regression tasks.
It uses a unique 'random subspace' approach for feature selection in each tree.
The difference in tree construction between XGBoost and LightGBM is that XGBoost generally uses a level-wise (BFS) approach, while LightGBM employs a leaf-wise (DFS) approach. What does the leaf-wise approach prioritize?
The parallel processing of multiple data columns simultaneously.
Minimizing the total number of splits across all trees.
Splitting the leaf node that will result in the maximum reduction in loss (highest gain).
Growing the tree in a highly balanced, symmetrical structure.
For a machine learning task involving a very large dataset with many features, which boosting algorithm is generally recommended for its superior speed and memory efficiency?
Random Forest
XGBoost
LightGBM
AdaBoost
What is the primary characteristic of the input data that necessitates the use of the Yeo-Johnson transformation over the Box-Cox transformation?
The data exhibits significant heteroscedasticity (non-constant variance).
The data is highly sparse and high-dimensional.
The data contains zero or negative values.
The data follows a bimodal or multimodal distribution.
The main machine learning objective for applying a Box-Cox or Yeo-Johnson transformation is to:
Transform the data distribution to better approximate a Gaussian (Normal) distribution.
Scale all feature values into a fixed range like [0, 1].
Convert continuous features into discrete, categorical features.
.Identify and remove correlated features from the dataset.
In Support Vector Machines (SVM), the Kernel Trick is used to overcome which challenge?
The need to balance the bias-variance trade-off in the final model.
The requirement for SVM to process only numerical features.
The inability of SVM to handle large datasets due to memory constraints.
The non-linear separability of data in the original feature space.
In an SVM model, the Support Vectors are defined as:
The data points that lie closest to the separating hyperplane and define the margin boundaries.
The features that have the highest importance score for the classification task.
The samples that are consistently misclassified, regardless of the kernel used.
All data points that are correctly classified by the Maximum Marginal Hyperplane.
The Naive Bayes Classifier is named 'Naive' because it makes a strong, often unrealistic, assumption that:
All feature values must follow a strict Gaussian (Normal) distribution.
The model can only be applied to binary classification problems.
The classifier must be trained on very small datasets to be effective.
All features are conditionally independent of each other, given the class variable.
Despite its 'Naive' assumption, the classifier performs exceptionally well in applications like spam filtering and sentiment analysis. This effectiveness is largely due to:
Its simplicity and computational efficiency, making it robust and fast for high-dimensional feature spaces (like word counts).
The feature sets (like word counts) being naturally independent in text data.
Its capability to implicitly perform feature scaling and selection before classification.
Its reliance on complex iterative optimization algorithms to minimize error.
In the K-Nearest Neighbors (KNN) algorithm for a classification task, how is the class of a new data point determined?
By maximizing the distance between the new point and the hyperplane.
By selecting the class that receives the majority vote among the K nearest neighbors.
By calculating the average feature values of the K nearest neighbors.
By predicting the class with the highest prior probability in the training data.
What is the primary risk associated with choosing a very small value for K (e.g., K=1) in the K-Nearest Neighbors algorithm?
The model becomes highly sensitive to noisy data or outliers, leading to high variance and overfitting.
The model becomes computationally simpler, ignoring features with low importance.
The model is likely to underfit, creating a very smooth decision boundary.
The model becomes too slow during the training phase (a 'lazy' model problem).
Why is feature scaling (e.g., standardization or normalization) considered a critical preprocessing step for the KNN algorithm?
To prevent features with a large magnitude or range from disproportionately dominating the distance calculations.
To reduce the risk of the model suffering from the 'curse of dimensionality'.
To increase the computational speed of the distance calculations.
To ensure the algorithm can handle categorical features correctly.
Pease write your Student Id and Name.
