SSL Exam

Which are the main differences when applying maximum likelihood for a single Gaussian versus a mixture of Gaussians?

• A. Overfitting can only happen for the single Gaussian model

• B. The mixture model is prone to singularities

• C. ML treatment for the mixture model can be solved in closed form, leading to a simple solution

• D. The mixture model may provide a better fit, depending on the dataset

Regularized least squares for a linear regression model is similar to a Bayesian treatment for regression when:

• A. The prior for the weights is a Dirichlet distribution

• The prior for the weights is any type of Gaussian

• The prior for the weights is a simple zero mean isotropic Gaussian

• D. There is no link between regularized regression models and Bayesian models for regression

Which of the following option(s) is / are true?

• A. You need to initialize parameters in PCA

• B. You don't need to initialize parameters in PCA

• C. PCA can be trapped into local minima problem

• D. PCA can't be trapped into local minima problem

 

  Regarding bias and variance, which of the following statements are true?  

• A. Models which overfit have a high bias.

• B. Models which overfit have a low bias.

• C. Models which underfit have a high variance.

• D. Models which underfit have a low variance

Which of the following is true about generative models?  

• A. They capture the joint probability

• B. The perceptron is a generative model

• C. Generative models can be used for classification

• D. They capture the conditional probability

Compared to the variance of the Maximum Likelihood Estimate (MLE), the variance of the Maximum A Posteriori (MAP) estimate is:  

• Higher

• Same

• Lower

• D. it could be any of the above

 

  How can we demonstrate that a function is a valid kernel? 

• A. We need to be able to express it as a dot product in a latent feature space, maybe with an infinite number of dimensions

• B. We need to be able to express it as a dot product in a latent feature space, but with a finite number of dimensions

• C. We show that the Gram matrix contains only positive elements

• D. We show that the design matrix is positive semi-definite

Which of the following statements is true about the MAX-SUM algorithm  

• Is derived from the SUM-PRODUCT algorithm

• Involves operations similar to message passing from leaves to root

• Is an exact inference algorithm for finding marginals applicable to tree-structured graphs

• Is an exact inference algorithm for finding x that maximizes p(x), along with this maximum value

• Is independent of which node is selected as the root of the algorithm

What can help to reduce overfitting in an SVM classifier?  

• High-degree polynomial features

• Setting a very low learning rate

• Use of slack variables

• Normalizing the data

Given a k-NN classifier, which one of the following statements is true?

• The more examples are used for classifying an example, the higher accuracy we obtain

• The more attributes we use to describe the examples the more difficult is to obtain high accuracy

• The most costly part of this method is to learn the model

• We can use KNN for classification only