MODEL SELECTION

What is the main goal of model selection?

• Add as many predictors as possible

• Choose the simplest model possible

• Balance accuracy and complexity

• Use all available data

Which method can completely remove some predictors?

• Ridge Regression

• Lasso Regression

• PCR

• Subset Selection

When should you use Ridge Regression?

• When predictors are independent

• When predictors are strongly correlated

• When you want to remove variables

• When you have few predictors

Which method creates new variables by combining existing ones?

• PCR and PLS

• Ridge

• Lasso

• Subset Selection

What is Cross-Validation used for?

• To increase sample size

• To estimate model performance

• To normalize variables

• To reduce dimensionality

In the marketing example, why was Ridge Regression used?

• To remove unimportant predictors

• To keep all small signals and shrink them

• To combine predictors into components

• To make the model interpretable

Which method is best when we believe only a few predictors are truly important?

• Ridge Regression

• PCR

• Lasso Regression

• PLS

What does AIC/BIC help with?

• Splitting data into training and testing

• Comparing model quality

• Making predictions

• Scaling the data

What is a key weakness of Best Subset Selection?

• It becomes too slow with many variables

• It always underfits the model

• It removes too many predictors

• It ignores correlation

PLS is different from PCR because it:

• Ignores the response variable

• Removes all predictors

• Considers the response when creating components

• Always removes collinearity