Supervised Learning II

What is the purpose of GridSearchCV in machine learning?

• To evaluate model performance using classification metrics

• To tune hyperparameters using an exhaustive search

• To preprocess data using feature scaling techniques

• To train a model using cross-validation

How does k-fold cross-validation work in GridSearchCV?

• It divides the dataset into k equal parts, each used as a separate validation set

• It performs a grid search on k different subsets of hyperparameters

• It trains the model k times, each time using a different subset of the data for validation

• It evaluates the model on k different metrics and selects the best combination

What is a decision tree in machine learning?

• A tree-shaped structure used to represent decision rules

• A method for feature selection

• A type of ensemble learning algorithm

• A graphical representation of the data distribution

What is entropy used for in decision trees?

• To measure the impurity of a node

• To calculate the information gain for splitting nodes

• To prune the tree and prevent overfitting

• To determine the optimal number of features

How does bagging differ from boosting?

• Bagging trains multiple models sequentially, while boosting trains them in parallel

• Bagging uses a single model to make predictions, while boosting uses an ensemble of models

• Bagging focuses on reducing model variance, while boosting focuses on reducing bias

• Bagging combines weak learners to create a strong model, while boosting combines strong learners

What is a Random Forest in machine learning?

• A linear regression model for predicting continuous outcomes

• A dimensionality reduction technique for high-dimensional data

• A clustering algorithm used for unsupervised learning

• An ensemble learning algorithm that combines multiple decision trees

How does Random Forest reduce overfitting compared to a single decision tree?

• By training each decision tree on a different subset of features

• By averaging the predictions of multiple decision trees

• By limiting the maximum depth of each decision tree

• By using majority voting to make predictions

What is the purpose of feature bagging in Random Forest?

• To reduce the correlation between individual decision trees

• To select the most informative features for model training

• To create diverse decision trees by training them on different subsets of data

• To speed up the training process by parallelizing computations

How does an SVM find the optimal decision boundary?

• By maximizing the margin between support vectors

• By selecting the features with the highest weights

• By iteratively updating the model parameters using gradient descent

• By minimizing the number of misclassified data points

What is the kernel trick in SVM?

• A technique for regularizing the SVM to prevent overfitting

• A preprocessing step for scaling features to a standard range

• A way to combine multiple SVM models using ensemble learning

• A method for transforming data into a higher-dimensional space to make it linearly separable