Bias in Surveys: Question Bank

Critique the Survey:

Comment on whether or not this survey exhibits one or more particular types of bias, and explain your reasoning:

TITLE: Survey on Exercise Habits Among Oceana High School Students. OBJECTIVE: To examine the exercise routines and preferences of students at Oceana High School in Pacifica, CA. METHODOLOGY: The survey was conducted entirely online, distributed through email invitations to the student mailing list and promoted on fitness-related social media groups popular among students. QUESTIONS: (a) Types of exercise activities participated in. (b) Frequency and duration of workouts. (c) Preferred locations for exercising (e.g., school gym, parks, home). (d) Demographic information such as grade level and extracurricular involvement.

Answer explanation

This survey exhibits undercoverage bias because it relies on voluntary participation and targets students who are active in fitness-related social media groups. As a result, students who are less interested in fitness or do not actively engage with these platforms are less likely to respond, leading to an overrepresentation of students who are already interested in exercise and potentially skewing the results to reflect higher levels of activity than the general student population.

Critique the Survey:

Comment on whether or not this survey exhibits bias, and explain your reasoning:

TITLE: Survey on Household Energy Usage. OBJECTIVE: To assess the energy consumption patterns of households in Pacifica. METHODOLOGY: A random sample of 500 Pacifica households was selected using the city’s utility records. Surveys were distributed via postal mail to ensure all households had an equal chance of participation. Follow-up reminders were sent to maximize response rates. QUESTIONS: (a) Monthly energy usage, (b) types of energy sources used, (c) energy-saving practices, (d) and demographic information such as household size and income.

Critique the Survey:

Comment on whether or not this survey exhibits bias, and explain your reasoning:

TITLE: Survey on Workplace Satisfaction. OBJECTIVE: To evaluate the job satisfaction levels among employees. METHODOLOGY: The survey was handed out to employees during lunch breaks at the main office building. Participation was limited to those present in the office on the survey days. QUESTIONS: (a) Job satisfaction ratings, (b) work-life balance, (c) relationship with supervisors, and (d) demographic information such as age and department.

Critique the Survey:

Comment on whether or not this survey exhibits bias, and explain your reasoning:

TITLE: Survey on Local Voting Preferences. OBJECTIVE: To understand the voting preferences of residents in the town. METHODOLOGY: A questionnaire was mailed to 1,000 registered voters. Out of these, only 200 completed and returned the survey. QUESTIONS: (a) Preferred candidates in the upcoming election, (b) Key issues influencing their vote, (c) and Demographic information such as age and income

Critique the Survey:

Comment on whether or not this survey exhibits bias, and explain your reasoning:

TITLE: Survey on Favorite Movie Genres. OBJECTIVE: To identify the most popular movie genres among community members. METHODOLOGY: The survey was posted on a public online forum and shared via social media platforms, inviting anyone interested to participate. There was no limit on the number of responses. QUESTIONS: (a) Favorite movie genres (e.g., action, comedy, drama, horror), (b) frequency of movie watching, (c) preferred platforms for watching movies, and (d) demographic information such as age and gender.

Scenario & Question

What percent of Americans wash their hands after using the bathroom? It depends on how you collect the data. In a telephone survey of 1006 U.S. adults, 96% said they always wash their hands after using a public restroom. An observational study of 6028 adults in public restrooms told a different story: Only 85% of those observed washed their hands after using the restroom.

Explain why the results of the two studies are so different.

Scenario & Question

Three hundred people participated in a free 12-week weight-loss course at a community health clinic. After one year, administrators emailed each of the 300 participants to see how much weight they had lost since the end of the course. Only 56 participants responded to the survey. The mean weight loss for this sample was 13.6 pounds.

Describe how nonresponse might lead to bias in this study. Is the mean amount of weight loss in the sample likely to be greater than or less than the mean for all 300 participants in the course?

Scenario & Question

How long before school starts do students get out of bed, on average? Administrators survey a random sample of students on each school bus one morning.

Describe how undercoverage might lead to bias in this study. Is the average amount of time in the sample likely greater than or less than the average for all students at the school?

Scenario & Question

A farmer brings many crates of oranges to a juice company each week. A company inspector looks at 10 oranges from the top of each crate before deciding whether to buy all the oranges.

Explain why this sampling method is biased. Is the proportion of damaged oranges in the sample likely greater than or less than the proportion of all oranges in the crate that are damaged?

Scenario & Question

The athletic department at El Dorado Community College wants to learn more about the physical fitness of students at the school, including the number of push-ups that students can perform. To obtain their sample, the athletic department sets up a booth in the center of campus with a sign that says "Free Physical Fitness Testing".

Explain why this sampling method is biased. Is the mean number of push-ups for the members of their sample likely to be greater than or less than the mean number of push-ups for the entire student body?