FIT2086 Week 7

What is the primary objective when fitting a linear regression model using the least squares method?

• • To maximize the correlation between the independent variables.

• • To minimize the number of predictors in the model.

• To minimize the sum of the squares of the residuals

• To maximize the R-squared value

Which of the following scenarios is indicative of an overfitting model?

• The model performs well on the training data but poorly on new, unseen data

• The model has low accuracy on both training and validation data.
• The model shows consistent performance across different datasets.
• The model performs equally well on both training and validation data.

What does a lower AIC value indicate when comparing multiple models?

• The model is simpler and underfits the data.

• A lower AIC value indicates a worse model fit.
• The model has a better fit to the data with a trade-off for complexity.

• The model is less likely to include irrelevant predictors.

You perform backward selection on a dataset using the following R command:

final_model <- step(lm(y ~ x1 + x2 + x3 + x4 + x5), direction = "backward")

Afterward, you observe that the final model includes only x1 and x4. Which of the following statements is most accurate about this final model?

• The predictors x2, x3, and x5 were removed because their inclusion did not significantly reduce the AIC.

• The predictors x2, x3, and x5 were removed because they were likely correlated with y.

• The final model suggests that x1 and x4 have no relationship with y.
• The model's performance on the training data will improve, but it may not generalize well to new data.

After performing stepwise regression, you notice the final model has an AIC close to the original model, but the number of predictors is reduced by half. Which of the following is a valid interpretation?

• The reduced model likely has higher predictive accuracy due to fewer predictors.

• The original model was overfitting the data, and the reduced model corrects this by removing noise.

• The final model is likely over-simplified, as it removed predictors that might still be relevant.

• The reduced model has a better trade-off between complexity and fit, which could improve its generalizability.

During model selection, you perform a t-test for the hypothesis H₀: β_j = 0 for each predictor. Which of the following scenarios would suggest that x_j should be removed from the model?

• The p-value for βj​ is greater than the significance level,

• xj should be removed if the coefficient βj is significantly positive.
• The p-value for βj is less than the significance level.

• The predictor xj has a low correlation with the response variable y.

In a binary classification problem, if the ROC curve for a logistic regression model is a diagonal line from (0,0) to (1,1), what does this imply about the model's performance?

• The model is no better than random guessing.

• The model has perfect predictive ability.

• The model has a high true positive rate.

• The model's predictions are perfectly calibrated.

Given a logistic regression model with predicted probabilities, which scenario would result in a high log loss value?

• The model predicts probabilities close to 0.5 for most instances.

• The model predicts probabilities close to 1 or 0, aligning well with the actual labels.

• The model's predicted probabilities are well-calibrated

• The model predicts a positive class with a probability of 0.9 when the actual label is 0.

Answer explanation

Logarithmic loss (log loss) penalizes predictions that are both confident and incorrect. If a model predicts a probability of 0.9 for the positive class when the actual label is 0, it results in a high log loss because the model is confident in its wrong prediction. This heavy penalty increases the overall log loss value.

Which of the following metrics is commonly used to evaluate the interpretability of a logistic regression model?

• Mean squared error
• R-squared value
• AUC (Area Under the Curve)

Answer explanation

• AUC (Area Under the Curve) is a metric that summarizes the performance of a classification model at different thresholds, making it a common choice to evaluate the overall performance of a logistic regression model.

• Mean Squared Error (MSE) and R² are more applicable to regression problems rather than classification and is not relevant for logistic regression.

In logistic regression, the log odds of the event can be expressed as:

• All of the above.