Intro to ML: Neural Networks Lecture 1 Part 1

A student is trying to predict the price of cars based on the car’s features e.g. number of owners, milage, condition, brand and age. What type of problem is this?

• Classification

• Regression

• Logistic Regression

• None of the above

Answer explanation

Classification requires discrete labels. Regression predicts points along a continuous scale. Logistic regression passes the output through a sigmoid function, mapping the output between 0 and 1. This is used for binary classification.

A lawyer is trying to predict whether a particular judge will grant bail or not for a suspect. The lawyer has a dataset detailing all the cases which the judge has presided over, where the suspects features such as age, number of prior convictions and severity of crime are detailed along with whether they were granted bail or not. Which technique from the following is best suited to the problem?

• Linear Regression

• Logistic Regression

• None of the above

Answer explanation

This is a binary classification task, therefore the sigmoid output of logistic regression (which maps outputs between 0 and 1) will suit well.

Is the following statement True or False? Gradient descent is a handy technique which allows us to use non differentiable functions/models and update them iteratively towards an optimum. If the function had been differentiable we could have simply found the exact solution to the problem by setting the derivative to 0 and rearrange to find the optimal parameters.

• True

• False

Answer explanation

We need to be able to differentiate the function to know which direction in the parameter space to move/step. The reason why we use gradient descent rather than finding the exact solution is that finding the exact solution requires matrix inversion which has cubic complexity.

Is the following statement True or False? Within the framework of linear regression, only linear functions can be modelled.

• True

• False

Answer explanation

We can perform a series of transformations on our input x. For example:

$y\ =\overline{\phi}^T\left(x\right)\overline{\theta}+b$
where $\overline{\phi}$ lifts the one dimensional space into a K dimensional space, a common example of this is detailed below. Through this, polynomials of degree $\le$ K-1 can be modelled.

Which of the following can be used to evaluate the performance of a linear regression model?

• ROC-AUC

• Accuracy

• Log loss

• Mean squared error

Answer explanation

Log loss is a measure of how close the model's predicted probability of a class is to the true value/label. ROC-AUC is a measure of the degree to which a model can distinguish between two classes. Accuracy is the degree to which the predicted classes match the true classes. Only mean squared error from these options can be applied to a regression task. Mean squared error is the expected distance between the models predictions and the true values.

Answer explanation