Classification_G1

Why is PCA sometimes used as a preprocessing step before regression?

• To reduce overfitting by removing poorly predictive dimensions.

• To make computation faster by reducing the dimensionality of the data

• For inference and scientific discovery, we prefer features that are not axis-aligned.

• To expose information missing from the input data.

What tends to be true about increasing the k in k-nearest neighbors?

• The decision boundary tends to get smoother

• As the number of sample points approaches infinity (with n/k → ∞), the error rate approaches less than twice the Bayes risk (assuming training and test points are drawn independently from the same distribution).

• The variance tends to increase.

• The bias tends to increase.

Which of the following are true about decision trees?

• All the leaves must be pure.

• Pruning usually achieves better test accuracy than stopping early.

• The tree depth never exceeds O(log n) for n sample points.

• They can be used only for classification.

Which of the following are true for k-nearest neighbor classification? (Hint: multiple statements may be true)

• It is more likely to overfit with k = 1 (1-NN) than with k = 1,000 (1,000-NN)

• The optimal running time to classify a point with k-NN grows linearly with k

• In very high dimensions, exhaustively checking every training point is often faster than any widely used competing exact k-NN query algorithm

• If you have enough training points drawn from the same distribution as the test points, k-NN can achieve accuracy almost as good as the Bayes decision rule

What is the number of parameters needed to represent a Naive Bayes classifier with n Boolean variables and a Boolean label ?

Are these statements true or false?

(A) When we use PCA, we need data to be labelled.

(B) PCA extracts the variance structure from high dimensional data such that the variance of projected data is minimized.

• True, True

• True, False

• False, True

• False, False

The following statement is true/false?

Gain ratio is less biased towards tests with many outcomes than InfoGain.

• True

• False

The following statement is true/false?

The range of InfoGain is [0, 1].

• true

• false

Given m data points, the training error converges to the true error as m → ∞.

• true

• false

No classifier can do better than a naive Bayes classifier if the distribution of the data is known.

• true

• false