Errors and Powers

Errors and Powers

AP Stat - Chapter 9 - Third Section

When we draw a conclusion from a significance test, we hope our conclusion will be correct. But sometimes it will be wrong.

• There are two types of mistakes we can make:

• Type I error or a Type II error.

Type I error 

A Type I error occurs if we reject H₀ when H₀ is true. That is, the data give convincing evidence that H_a is true when it really isn’t.

​

So, in the example, there truly is no fire, but the fire alarm is telling you there is one. ​This is a False Positive (you get a result but you shouldn't)

Type I Error:

You shouldn't reject, but you do.​

False Positive (You get evidence to reject, but its WRONG).

Type II error

A Type II error occurs if we fail to reject H₀ when H_a is true. That is, the data do not give convincing evidence that H_a is true when it really is.

​

In the example, there truly is a fire so we should reject, but our evidence tells us there isn't a fire. This is a false negative.​

Type II Error:

You should reject, but you don't.​

False Negative (You get don't get evidence to reject, but you should have. Data is bad!).

Let's Let Sal Tell Us About Errors

Prepare for 5 minutes of a video!

Multiple Choice

A type I error involves a researcher wrongly accepting which hypothesis?

• Alternative

• Null

Multiple Choice

Which of the following refers to a type I error?

• When a researcher wrongly accepts the results as being significant.

• When a researcher wrongly accepts the results as not being significant.

Multiple Choice

Which of the following refers to a type II error?

• When a researcher wrongly accepts the results as being significant.

• When a researcher wrongly accepts the results as not being significant.

Multiple Choice

A type II error involves a researcher wrongly accepting which hypothesis?

• Alternative

• Null

TYPE I ERROR PROBABILITY

The probability of making a Type I error in a significance test is equal to the significance level α.

Decrease Type I Error

We can decrease the probability of making a Type I error in a significance test by using a smaller significance level

Trade-off

There is a trade-off between P(Type I error) and  P(Type II error): as one increases, the other decreases, assuming everything else remains the same.

Alpha Issues

If we make it more difficult to reject H₀ by decreasing α, we increase the probability that we will not find convincing evidence for H_a when it is true. That’s why it is important to consider the possible consequences of each type of error before choosing a significance level.

Multiple Choice

A car manufacturer advertises that its new subcompact models get 47 mpg. If μ is the mileage of these cars, what could be the null and alternate hypothesis if we wanted to check if the car's mpg is overrated? 

• H₀ = 47
  H_a = 47
• H₀ = 47
  H_a < 47
• H₀ = 47
  H_a > 47
• H₀ = 47
  H_a ≠ 47

Multiple Choice

When p-value is greater than alpha we:

• Accept Ho
• Accept Ha
• Fail to reject Ho
• Reject Ho

Multiple Choice

What does it mean when a test is statistically significant?

• it has reached alpha level status
• the test statistic had a P-value higher than the alpha level
• the test statistic had a P-value lower than the alpha level
• it is important in practical terms

Power

​

The power of a test is the probability that the test will find convincing evidence for H_a when a specific alternative value of the parameter is true.

​

Multiple Choice

A test is made of H₀ : µ = 17 versus H₁ : µ < 17 is performed using α = 0.01 significance level. The value of the test statistic is z = -2.68 and it is concluded that H₀ is rejected.

If the true value of µ is 10, is the conclusion a Type I Error, Type II error, or a correct decision?

• Type I

• Type II

• Correct Decision

Multiple Choice

Which of the following is the FIRST step of a hypothesis test?

• Check conditions

• Calculate the test statistic

• Evaluate the p value

• State your null & alternative hypotheses

Multiple Choice

Suppose the P-value for a hypothesis test is 0.0304. Using a = 0.05, what is the appropriate conclusion?

• Reject the null hypothesis

• Reject the alternative hypothesis

• Fail to reject the null hypothesis

• Fail to reject the alternative hypothesis

Type II Error is determined by β

Power is the probability that we find convincing evidence the alternative hypothesis is true, given that the alternative hypothesis really is true. Power = 1-β

Want to reduce both errors at the same time?

• Increase sample size!

Multiple Choice

What is the Power of a test?

• A strong force
• The probability that we correctly reject a false null.
• The probability that we correctly accept a true null.
• How tough it is.

Multiple Choice

A test is made of H₀ : µ = 17 versus H₁ : µ < 17 is performed using α = 0.01 significance level. The value of the test statistic is z = -2.68 and it is concluded that H₀ is rejected.

If the true value of µ is 17, is the conclusion a Type I Error, Type II error, or a correct decision?

• Type I

• Type II

• Correct Decision

Multiple Choice

How can you reduce both Type I and Type II errors?

• it can't be reduced
• redo the tests
• increase the sample size
• tamper with the data

Multiple Select

Asha owns a car-wash and is trying to decide whether or not to purchase a vending machine so customers can buy coffee while they wait. She'll get the machine if she's convinced that more than 30%   of her customers would buy coffee. She plans on taking a random sample of n customers and asking them whether or not they would buy coffee from the machine, and she'll then do a significance test using $$\alpha\ =\ 0.05$$   to see if the sample proportion who say "yes" is significantly greater than 30%  .

Suppose that in reality, it is actually 33% of her customers that would buy coffee.

Which of the changes below would result in the highest power for her test?

• Increase the Sample Size to 200

• Increase the Sample Size to 50

• Increase the Significance Level to 0.01

• Decrease the Significance Level to 0.10

Open Ended

Tell me something thatt makes sense about Type I & II errors and power.

What do you still need?

All done!

Error Practice: Khan Academy.
Power Practice: Khan Academy