Concept Drift Testing Quiz

What is an example of concept drift in real-world scenarios?

• A weather prediction model failing due to seasonal changes.

• A housing price model failing because buyer preferences shifted toward 'Remote Work friendly' homes after the pandemic.

• A stock market model failing due to consistent trends in stock prices.

• A traffic prediction model failing due to unchanged road conditions.

Which type of concept drift involves an old concept gradually being replaced by a new one over time?

• Sudden drift

• Gradual drift

• Incremental drift

• Reoccurring concepts

What does "Data Drift Testing" aim to identify?

• Whether the distribution of input data has shifted from what the model saw during training.

• Whether the model is overfitting to the training data.

• Whether the model is underfitting to the test data.

• Whether the model's accuracy has improved over time.

What is "Covariate Shift" in the context of data drift testing?

• A complete replacement of the training data with new data.

• A gradual improvement in the model's performance over time.

Which type of data drift occurs when a new concept replaces an old one over a period of time?

• Sudden Drift

• Gradual Drift

• Incremental Drift

• Reoccurring Concepts

Which of the following is an example of a performance metric that can trigger automated retraining?

• Sharpe Ratio > 2.0

• Accuracy < 85% or F1-Score < 0.7

• Model size > 1GB

• Training time > 10 hours

What happens when integration with CI/CD triggers are met?

• The system stops all operations.

• The system pulls new data, retrains, and runs an automated 'Golden Set' test before deployment.

• The system sends an alert to the user for manual retraining.

• The system deletes the existing model.

Why is traditional accuracy considered useless in fraud detection according to the case study?

• It is too complex to calculate.

• It does not account for the imbalance in fraud cases.

• It requires expensive tools to measure.

• It is only applicable to balanced datasets.

What is the primary focus of the Precision-Recall metric in fraud testing?

• Testing if the model fails in specific cities or countries.

• Finding fraud without annoying real users.

• Identifying phishing tactics in local regions.

• Increasing the number of fraud cases detected.

What does Regional Slicing test in fraud detection models?

• The ability to detect phishing tactics.

• The model's performance in specific cities or countries.

• The recall rate of the model globally.

• The number of fake accounts detected.