Understanding P-Values in Psychology

What does a p value mean in psychology research?

• The chance that the hypothesis is right

• The chance of getting results like these if the null hypothesis is true

• The percent of people who got the expected result

• How big the difference is between two groups

What is the conventional threshold for statistical significance commonly used in psychological research?

• p < 0.10

• p < 0.01

• p < 0.05

• p < 0.001

Which of the following correctly defines the null hypothesis?

• A statement predicting that a significant difference or relationship exists between variables

• A statement predicting that the independent variable will have a large effect

• A statement predicting that there is no significant difference or relationship between variables

• A statement predicting that the results will support the research hypothesis

If a researcher obtains a p value of 0.03, what does this indicate?

• There is a 3% chance the null hypothesis is true

• There is a 97% chance the results are meaningful

• The result is statistically significant at the p < 0.05 level

• The result is not statistically significant

A psychology student conducts an experiment and obtains a p value of 0.08. Which of the following conclusions is most appropriate?

• The result is statistically significant at the p < 0.05 level, so the null hypothesis is rejected

• The result is not statistically significant at the p < 0.05 level, so the null hypothesis is retained

• The result proves the null hypothesis is true

• The result confirms the research hypothesis is correct

A researcher finds p = 0.001 in their study. How does this compare to a result of p = 0.04 in terms of statistical significance?

• Both results are equally significant because both are below 0.05

• p = 0.001 provides stronger evidence against the null hypothesis than p = 0.04

• p = 0.04 is more significant because it is closer to 0.05

• p = 0.001 is less significant because it is a smaller number

Which of the following statements correctly explains the relationship between sample size and the p value?

• Larger sample sizes always produce smaller p values regardless of the effect

• Smaller sample sizes tend to produce more reliable p values

• Larger sample sizes increase statistical power, making it more likely to detect a true effect and obtain a smaller p value

• Sample size has no influence on the p value obtained

In a psychology experiment, a researcher is comparing the mean anxiety scores of two groups. The p value obtained is 0.04. Which of the following correctly applies this finding?

• The null hypothesis is accepted because the p value is less than 0.05

• The result is statistically significant, and the null hypothesis is rejected at the p < 0.05 level

• The result is not statistically significant because 0.04 is very close to 0.05

• The research hypothesis is proven to be correct

A researcher conducts a study on the effect of sleep deprivation on memory recall. They obtain a p value of 0.06. A colleague suggests repeating the study with a larger sample. Explain why increasing the sample size might change the outcome.

• A larger sample would automatically make the p value smaller, regardless of the actual effect

• A larger sample reduces variability and increases statistical power, which may allow a true effect to be detected with a p value below 0.05

• A larger sample would increase the p value, making the result more significant

• Sample size does not affect the p value in psychological research

A psychologist conducts two separate studies on the same topic. Study A has a sample size of 20 and obtains p = 0.04. Study B has a sample size of 200 and obtains p = 0.04. Which study provides more reliable evidence and why?

• Study A, because smaller samples are more controlled and less likely to include confounding variables

• Both studies are equally reliable because they obtained the same p value

• Study B, because a larger sample size reduces the influence of random error and increases the generalisability of the findings

• Study A, because a smaller sample is easier to replicate