Reviewing Experiments and Observational Studies

An advertising agency is testing the effectiveness of a new commercial. They are trying out the commercial in three different time lengths (15, 30, and 60 seconds) and two formats (animated vs. live action). Each test audience will rate the product after seeing the commercial shown in one way. Which description is correct?

• An experiment with 2 factors (length and format), one having 3 levels (15, 30, 60) and the other having 2 levels (animated and live) for 6 treatments.

• An experiment with 2 factors (length and format) at 5 levels (15, 30, 60, animated and live) for 5 treatments.

• An experiment with 1 factor (length) blocked by format with 3 levels (15, 30, 60).

• A sample survey with two questions and one response.

• A sample stratified by length and format.

Tags

CCSS.HSS.IC.B.3

CCSS.HSS.IC.B.5

A school newspaper is going to investigate students' perceptions regarding the campus security guards. They plan to survey 100 students at the school. The editor-in-chief tells his staff that he is worried about undercoverage bias created by chronically truant students. His concern is...

• unwarranted because some students may refuse to participate and that is inevitable.

• justified because chronically truant students will be difficult to survey and will probably have a disproportionately negative view of security guards.

• unwarranted because students who cannot be surveyed are not a part of the population of interest.

• unwarranted because students who cannot be surveyed are not a part of the sample.

• justified because people need to be forced to answer a surey in order for the results to be valid.

Tags

CCSS.HSS.IC.A.1

CCSS.HSS.IC.B.3

CCSS.HSS.IC.B.4

Which is one of the main differences between experiments and observational studies?

• There is a response variable in an experiment, but not in an observational study.

• There is at least one explanatory variable in an experiment, but not in an observational study.

• An experiment requires blocking, while an observational study does not.

• An experiment can be used to establish a causal relationship, but an observational study cannot.

• Observational studies require larger samples than experiments.

Tags

CCSS.HSS.IC.B.3

In one study, researchers at McGill University recruited 127 people with high cholesterol and randomly split them into two groups. One took a probiotic supplement twice a day for nine weeks, while the second group took a placebo. The probiotic group saw their total cholesterol drop by 9 percent. In a different study, conducted in Britain, 80 volunteers were given probiotics for six weeks and then switched, later on, to a placebo - or vice versa (chosen at random). That study found no difference in cholesterol levels when the subjects took the supplement versus the placebo.

• The McGill study is a controlled randomized experiment, while the British study is a matched pairs experiment.

• The McGill study is a completely randomized experiment, while the British study is a matched pairs experiment.

• The results from the McGill study are more valid because it includes blinding while the British study does not.

• We have reason to believe that the design of one of these experiments was flawed since the results are contradictory.

• Conclusions from the British study can not be trusted since there is no control group.

Tags

CCSS.HSS.IC.B.3

CCSS.HSS.IC.B.5

In a Japanese study, "researchers look at 35 people with lower back pain who were enrolled in an aquatic exercise program, which included swimming and walking in a pool. Almost all of the patients showed improvements after six months, but the researchers found that those who participated at least twice weekly showed more significant improvement than those who went only once per week."

• This experiment proves that swimming causes a reduction in lower back pain.

• The study is a poorly designed experiment since there is no control group and there is no randomization.

• Conclusions based on this observational study are suspect since participation in the aquatic program may be confounded with other lifestyle behaviors that may cause the improvement in lower back health.

• Conclusions based on this observational study are suspect since a sample size of 35 is too small.

• The study is not an observational study since the aquatic program is a treatment.

Tags

CCSS.HSS.IC.B.3

CCSS.HSS.IC.B.6

CCSS.HSS.ID.C.9

Statistics teachers often debate the best order in which to teach topics. One group of teachers likes to teach design of studies first. Another group likes to begin with data analysis. To see which order is more effective in preparing students for the AP Exam, an experiment was proposed. A large group of teachers, each of whom teaches two sections of statistics, volunteered to be part of the experiment. Each teacher will randomly assign one of their classes to beign with design, and the other to begin with data analysis. Which is the correct description of this design?

• The experimental units are the classrooms, the blocks are the teachers, and the response variable is the difference in average AP Exam score for each of the teacher's classes.

• The experimental units are the teachers, there are no blocks, and the response variable is the average AP Exam score for each teacher.

• The experimental units are the individual students, the blocks are the classrooms, and the response variable is each individual student's AP Exam score.

• The experimental units are the individual students, the blocks are the teachers, and the response variable is the average AP Exam score for each classroom

• The experimental units are the order of the topics, the blocks are the teachers, and the response variable is the average AP Exam score for all students who used each order of topics.

Tags

CCSS.HSS.IC.B.3

CCSS.HSS.IC.B.5

A local news program decides to conduct a poll to see how their viewers feel about their programming format. At the end of each program during one week they ask viewers to call in and express their opinions. The station gives one number to dial if you like the new format, and another to dial if you do not like it. Which of the following is a correct characterization of this sampling approach?

• This sampling approach will be biased because people might dial the number incorrectly, and therefore not everyone will be correctly represented.

• This sampling approach will be biased because the people who call in will be those who have stronger feelings about the new format, and therefore the sample will not be representative of the population.

• This sampling approach will be unbiased because people can call in with either opinion, and therefore the sample will be representative of the population.

• This sampling approach will be unbiased because the station doesn't know who will call in, making it a random sample.

• It is impossible to tell whether this sampling approach will be biased or not because the station cannot predict who will call in.

Tags

CCSS.HSS.IC.A.1

CCSS.HSS.IC.B.3