Statistical Concepts Quiz

What does a spurious correlation illustrate?

• A real causal link

• A non-causal relationship driven by a hidden factor or coincidence

• A statistical anomaly that never repeats

• A purposeful manipulation of data

Which of these is not part of a strong hypothesis?

• What you are changing

• The metric you expect to impact

• The expected effect size

• The randomization criteria

“Reverse causality” is when you mistake the direction of cause and effect.

• True

• False

Why do we avoid feature‑level metrics like “clicks” as primary KPIs?

• They’re too hard to track

• They don’t directly tie to business impact

• They inflate sample‑size needs

• They require Chi‑square tests

Which statistical test is most appropriate for comparing conversion rates?

• Z‑test for proportions

• One‑sided t‑test

• Chi‑square test

• Two‑sided t‑test

What does an “underpowered test” mean?

• It has a high false‑positive rate

• It has a low chance of detecting a real effect

• It tests too many variations at once

• It inflates the significance level

Peeking at results before the test ends primarily increases…

• Type II errors (false negatives)

• Type I errors (false positives)

• Practical significance

• Test power

Statistical significance (p < 0.05) tells us that…

• The observed result is unlikely due to chance

• The effect is practically meaningful

• The test had enough power

• The confidence interval excludes zero

A 95% confidence interval of [+2%, +6%] means…

• There’s a 95% chance the true lift lies between +2% and +6%

• 95% of users saw a lift between +2% and +6%

• We’ll get this range in 95% of future tests

• The test had 95% power

“Statistical significance” is more important than “Practical significance.”

• True

• False