Metrics in Machine Learning

What can be said about the **precision** of this test? A test is used to diagnose a rare disease. Out of 1000 patients, 10 actually have the disease. The test correctly identifies 8 of these 10 patients but also incorrectly labels 50 healthy patients as diseased.

• High precision, because most predicted positives are true positives

• Low precision, because many predicted positives are false positives

• Precision equals recall

• Precision cannot be determined from the data

Which metric would best reflect the system’s ability to identify all spam emails? A spam filter identifies 90 spam emails out of 100 actual spam emails. It also incorrectly flags 30 legitimate emails as spam.

• Accuracy

• Precision

• Recall

• F1 Score

What is the approximate **recall** of the model? Out of 5000 transactions, only 20 are fraudulent. A model catches 15 frauds correctly but wrongly flags 10 legitimate transactions as fraud.

• 15/20 = 0.75

• 10/20 = 0.5

• 15/25 = 0.6

• 10/4990 ≈ 0.002

Calculate the **precision** of the model. A model predicted 100 emails as promotional, out of which 80 are truly promotional. It missed 20 promotional emails.

• 80/100 = 0.8

• 80/120 = 0.67

• 20/100 = 0.2

• 100/120 = 0.83

What is the **recall** of the model? Out of 500 images of cats, a model identifies 450 correctly and wrongly labels 50 dog images as cats.

• 450/500 = 0.9

• 450/500 + 50 = 0.9

• 50/500 = 0.1

• Cannot be determined

What is the **F1 score** of the model? (Recall = TP/(TP+FN), Precision = TP/(TP+FP)) A sentiment analysis model predicts 100 positive reviews, of which 70 are truly positive. It missed 30 positive reviews.

• 2*(0.7*0.7)/(0.7+0.7) = 0.7

• 2*(0.7*0.7)/(0.7+0.3) = 0.49

• 2*(0.7*0.7)/(0.7+0.7) = 0.7

• 2*(0.7*0.7)/(1.0) = 0.98

What is the **accuracy** of the model, assuming total applications = 500 and 300 were correctly rejected? A model predicts 200 credit approvals, with 180 correct approvals. It missed 20 valid applications.

• (180 + 300)/500 = 0.96

• 180/200 = 0.9

• 180/300 = 0.6

• Cannot determine from given info

Which statement is true? A test has 90% recall but only 50% precision.

• Test catches most actual cases but has many false positives

• Test catches few cases but false positives are low

• Test is balanced in recall and precision

• Test has low false negatives

Which metric decreases when false positives increase? A model identifies 400 faces correctly and mislabels 100 unknown faces as known.

• Recall

• Precision

• Accuracy

• None of these

What is the recall? Out of 1000 texts, 100 are positive. Model predicts 120 positives with 80 true positives.

• 80/100 = 0.8

• 80/120 = 0.67

• 120/1000 = 0.12

• Cannot tell