Observational Studies & Experimentation

Observational Studies & Experimentation

AP Statistics Chapter 12 Review

Observational Study vs. Experiment

An observational study is a process in which data is collected WITHOUT imposing treatments on participants.

An experiment is a process in which data is collected by randomly assigning treatments to participants.

Multiple Choice

A study took random sample of adults and asked them about their bedtime habits. The data showed that people who drank a cup of tea before bedtime were more likely to go to sleep earlier than those who didn't drink tea.

This is an example of an...

• Observational Study

• Experiment

Multiple Choice

Another study took a group of adults and randomly divided them into two groups. One group was told to drink tea every night for a week, while the other group was told not to drink tea that week. Researchers then compared when each group fell asleep.

This is an example of...

• Observational Study

• Experiment

Key Features of Experimentation

A factor is the independent variable, or the variable that the experimenter manipulates to create a response. The levels are the specific amounts/types of that factor. Treatments are the combinations of factors and levels.

The response variable is what the experimenter is recording.

Experimental units are who/what is receiving treatments...remember particiPLANTS?

Torn ACL

Having at least one 15-minute warm-up session per week resulted in a drastic reduction in tears in the ACL. In a study involving 4500 adolescent girl soccer players in Sweden, one group was randomly assigned to warm up with a neuromuscular exercise session and the other was assigned no exercise session. The group with the neuromuscular session had 64% fewer ACL tears than the control group.

What is the factor(s)? How many levels are there? What are the treatments?

Torn ACL Answer

Factor: Warm-Up

# of Levels: 2

Treatments: Neuromuscular Exercise Session, No Warm-Up Session

Torn ACL

Having at least one 15-minute warm-up session per week resulted in a drastic reduction in tears in the ACL. In a study involving 4500 adolescent girl soccer players in Sweden, one group was randomly assigned to warm up with a neuromuscular exercise session. This group had 64% fewer ACL tears than the control group.

What are the experimental units?

Torn ACL Answer

Experimental Units: 4500 adolescent girl soccer players

Torn ACL

Having at least one 15-minute warm-up session per week resulted in a drastic reduction in tears in the ACL. In a study involving 4500 adolescent girl soccer players in Sweden, one group was randomly assigned to warm up with a neuromuscular exercise session. This group had 64% fewer ACL tears than the control group.

What is the response variable?

Torn ACL Answer

Response Variable: Number of torn ACLs

Tossing & Turning

Is diet or exercise effective in combating insomnia? Some believe that cutting out desserts can help alleviate the problem, while others recommend exercise. Forty volunteers suffering from insomnia agreed to participate in a month-long test. Half were randomly assigned to a special no-dessert diet; the others continued desserts as usual. Half of the people in each of these groups were randomly assigned to an exercise program, while the others were assigned to not exercise. Those who ate no desserts and engaged in exercise showed the most improvement.

What is/are the factor(s)? How many levels are there? What are the treatments?

Tossing & Turning Answer

Factor(s): Diet & Exercise

# of Levels: Diet - 2 Exercise - 2

Treatments: No Desserts & Exercise, No Desserts & No Exercise, Desserts & Exercise, Desserts & No Exercise

Tossing & Turning

Is diet or exercise effective in combating insomnia? Some believe that cutting out desserts can help alleviate the problem, while others recommend exercise. Forty volunteers suffering from insomnia agreed to participate in a month-long test. Half were randomly assigned to a special no-dessert diet; the others continued desserts as usual. Half of the people in each of these groups were randomly assigned to an exercise program, while the others were assigned to not exercise. Those who ate no desserts and engaged in exercise showed the most improvement.

What are the experimental units?

Tossing & Turning Answer

Experimental Units: 40 Volunteers

Tossing & Turning

Is diet or exercise effective in combating insomnia? Some believe that cutting out desserts can help alleviate the problem, while others recommend exercise. Forty volunteers suffering from insomnia agreed to participate in a month-long test. Half were randomly assigned to a special no-dessert diet; the others continued desserts as usual. Half of the people in each of these groups were randomly assigned to an exercise program, while the others were assigned to not exercise. Those who ate no desserts and engaged in exercise showed the most improvement.

What is the response variable?

Tossing & Turning Answer

Response Variable: Improvement in Insomnia

Blinding & Placebos

There are two key groups in an experiment:

1. Those who could influence the results - experimental units, treatment administrators, etc.

2. Those who evaluate the results - judges, evaluators.

An experiment is single-blind when all the individuals in ONE of the groups are blinded. An experiment is double-blind when all the individuals in BOTH of the groups are blinded.

A placebo is a fake treatment that looks/feels/acts like a real treatment. ALWAYS use a placebo when possible.

Multiple Choice

30,000 volunteers participated in the Moderna COVID-19 vaccine trial. Half were randomly assigned to receive the active vaccine, while the other half received a placebo. Neither the patient, administrator, nor evaluator of health were aware of the specific treatment the patient received.

• Single-Blind

• Double-Blind

• No Blinding Utilized

Multiple Choice

You have 20 tomato plants. You randomly assigned half of the plants to get a dose of fertilizer, while the other half will not get fertilizer. After 3 months, judges will evaluate the tastiness of these tomatoes and will not be told which tomatoes received fertilizer.

• Single-Blind

• Double-Blind

• No Blinding Utilized

Torn ACL

Having at least one 15-minute warm-up session per week resulted in a drastic reduction in tears in the ACL. In a study involving 4500 adolescent girl soccer players in Sweden, one group was randomly assigned to warm up with a neuromuscular exercise session. This group had 64% fewer ACL tears than the control group.

Can this experiment be single-blinded? Double-blinded?

When should an experiment be blinded?

When the response variable involves opinion/judgement, it is always best to at least blind the evaluator.

The best experiments are always placebo-controlled, and double-blind.

Key Features of Experimental Design

1. Control

This is how we control KNOWN sources of variation! We must "even the playing field" across our groups so that the only difference between the groups is the treatment we administer.

2. Randomization

Random assignment of treatments is the ONLY tool we have against know/unknown bias.

3. Replication

We have to have more than one experimental unit in each group. The results of our experiment must happen multiple times (that's the replication part!)

Types of Experimental Design

1. Completely Randomized

Simple, gets the job done.

2. Block

Use blocking when you think there is something that makes groups similar and could affect the response variable! (age, sex, etc.)

3. Matched Pairs

More sophisticated block design. Match things up when you believe they are similar in most ways, so the only difference is the factor you control.

Multiple Choice

You are interested in the effects of a sensitive skin dog food. The participants include 2 toy breeds, 2 small breeds, 2 medium breeds, and 2 large breeds. You randomly assigned one of each size breed to the sensitive skin food, and the other to a regular food.

Identify the type of experimental design used.

• Completely Randomized

• Block

• Matched Pairs

Multiple Choice

Having at least one 15-minute warm-up session per week resulted in a drastic reduction in tears in the ACL. In a study involving 4500 adolescent girl soccer players in Sweden, one group was randomly assigned to warm up with a neuromuscular exercise session. This group had 64% fewer ACL tears than the control group.

Identify the type of experimental design used.

• Completely Randomized

• Block

• Matched Pairs

Multiple Choice

A new cholesterol lowering drug is being studied. The 112 volunteers are separated into groups based on their current exercise level: low, moderate, high. Half of the volunteers in each group are randomly assigned to receive the new drug, while the remaining half are assigned to receive a placebo.

Identify the type of experimental design used.

• Completely Randomized

• Block

• Matched Pairs

"Statistically Significant"

We always expect small differences between treatment groups, that's just life.

We say differences between treatment groups are statistically significant when differences are "large enough" and cannot reasonably be attributed to day-to-day variation