Introduction to Bias

​Survey Design Project

AP Stats Chapter 4

In this unit, we will be studying surveying methods.

You will work with a partner to collect data regarding a problem at school, and present your findings to a teacher and admin panel.

​

Survey Design

First, let's take a survey...

Poll

How much has Shawano’s increasingly strict and unnecessary cell phone policy negatively impacted your learning and mental health this year?

• Not at all

• Somewhat

• A Lot

Poll

Don’t you agree that the new grading policies have made it harder for responsible students to succeed?

• Yes

• Absolutely

Poll

How supportive and effective have your teachers been in preparing you for real-world success and college readiness?

• Extremely supportive and effective

• Somewhat supportive and effective

• Very supportive and effective

Poll

Given how stressful senior year already is, how overwhelmed do you feel by the excessive amount of homework assigned by most teachers?

• Slightly Overwhelmed

• Moderately Overwhelmed

• Extremely Overwhelmed

Poll

Seniors who take at least one AP class: How much more academically prepared do you feel than students who don’t challenge themselves with advanced coursework?

• Slightly more prepared

• Much more prepared

• Significantly more prepared

Something was wrong with these...

Open Ended

What frustrated you about the above survey?

BIAS!!!!

What do we already know?

Multiple Choice

Identify the sampling method:

Every fifth person boarding a plane is searched thoroughly.

• SRS

• Stratified

• Cluster

• Systematic

• Voluntary Response

Multiple Choice

Identify the sampling method:

At a local community College, five math classes are randomly selected out of 20 and all of the students from each class are interviewed.

• SRS

• Stratified

• Cluster

• Systematic

• Convenience

Multiple Choice

Identify the sampling method:

A researcher randomly selects and interviews fifty male and fifty female teachers.

• SRS

• Systematic

• Stratified

• Cluster

• Convenience

Multiple Choice

Identify the sampling method:

Based on 12,500 responses from 42,000 surveys sent to its alumni, a major university estimated that the annual salary of its alumni was 92,500.

• SRS

• Cluster

• Stratified

• Convenience

• Voluntary Response

Multiple Choice

Identify the sampling method:

A community college student interviews the first 100 students to enter the building to determine the percentage of students that own a car.

• SRS

• Stratified

• Cluster

• Convenience

• Voluntary Response

Multiple Choice

Identify the sampling method:

The names of 70 contestants are written on 70 cards, The cards are placed in a bag, and three names are picked from the bag.

• SRS

• Systematic

• Stratified

• Cluster

• Convenience

Multiple Choice

A company wants to know the opinion of a rural community on a proposed ballot initiative. Half of the community does not have internet access, so the company sends emails to the 380 members that do have internet access. Of those surveyed, 372 of the community members responded to the survey. Which of the following is the most significant source of bias in the survey?

• Voluntary Response Bias

• Undercoverage

• Nonresponse Bias

• Response Bias

Multiple Choice

A radio station is discussing a controversial policy enacted by the local police department. They poll listeners by having them call in to the radio station. Which of the following is the most significant source of bias?

• Leading questions on the survey

• Voluntary Response Bias

• Survivorship Bias

• Undercoverage

Multiple Choice

Dylan wants to know how the students at his school rate the cafeteria.  He surveyed his school's basketball team to rate the school cafeteria. Is this a biased sample of the school's population?

• Yes, because it is a convenience survey

• No, the basketball team has the smartest students

• Yes, because it's voluntary survey

• No, he should have asked the football team

Multiple Choice

Tanya wants to know the percent of people likely to attend the upcoming musical at school. She asks each person in the musical how many people they know who are coming to make a prediction. Why is this sample biased?

• Drama students are more likely to lie.

• Musicals are better attended than plays.

• The drama teacher was not asked.

• The sample is not random and not representative of the entire school population.

Multiple Choice

The School Dance Committee conducts a survey to find what type of music students would like to hear at the next dance. Which sampling method is least likely to result in a biased sample?

• Call the top 20% of the senior class. 

• Interview every 10^th student as they enter the school.

• Ask every 5^th person leaving a school jazz band concert.

• Set up a hip-hop only website where students can list their 3 favorite songs if they open up the link.

Multiple Choice

In order to estimate the proportion of students that text while driving, a school administrator selects a simple random sample of students from a list of all students at the school who have parking permits. The students are called to the office, one at a time. The administrator asks each student, “Do you text while driving, even though you are not supposed to?” Based on the survey, the administrator estimates that only 2% of students with parking passes text while driving. What potential bias is present in the design of this survey and what is the likely direction of the bias?

• The sample proportion is likely an underestimate of the population proportion due to response bias.

• The sample proportion is likely an underestimate of the population proportion due to undercoverage.

• The sample proportion is likely an underestimate of the population proportion due to nonresponse.

• The sample proportion is likely an overestimate of the population proportion due to response bias.

• The sample proportion is likely an overestimate of the population proportion due to undercoverage.

Multiple Choice

Volunteers for a politician's campaign called 100 randomly selected citizens to conduct a survey. Only 50 people agreed to answer the survey questions, and most of them were supporters of the candidate. This is an example of...

• Voluntary Response Bias

• Nonresponse Bias

• Undercoverage Bias

Multiple Choice

Determine if the following study is biased.

A senator wants to how she is doing. She sends a survey to 1000 registered voters in her party.

• Biased

• Unbiased

·       Leading question

·       Loaded or emotional language

·       Assumption built into question

·       Double-barreled question

·       Unbalanced response options

·       Social desirability bias

·       Sampling bias

·       Undercoverage

·       Response bias

​

Types of Bias

Let's revisit our initial survey

• ​loaded language

• assumes negative impact

• double-barreled: learning and mental health

• limited answer choices

Question 1:

Poll

How, if at all, has the school’s cell phone policy affected your learning this year?

• Negatively

• Not at all

• Positively

• Unsure

• ​leading question

• presumes agreement

• excludes disagreement option

• vague term

  “responsible students.”

Question 2:

Poll

How would you describe the impact of the new grading policies on student success?

• Very Negative

• Somewhat Negative

• No Impact

• Somewhat Positive

• Very Positive

• assumes preparation for "real-world success and college"

• double-barreled (supportive and effective)

• no negative options.

Question 3:

Poll

How well do you feel your teachers have prepared you for your plans after high school?

• Very Poorly

• Somewhat Poorly

• Neither Well nor Poorly

• Somewhat Well

• Very Well

• Loaded words (“excessive”)

• Assumption that homework is excessive

• Assumes stress and overwhelm

• No option for “not overwhelmed.”

• No negative options

Question 4:

Poll

How often do you feel overwhelmed by homework assignments?

• Never

• Rarely

• Sometimes

• Often

• Always

• Sampling bias
  Only AP seniors surveyed

• Loaded language

  “Don’t challenge themselves” pressures responses

• Presumes AP students are more prepared

• No neutral or “less prepared” option

• Social desirability bias Encourages students to rate themselves higher

Question 5:

Poll

How academically prepared do you feel compared to other seniors in your grade?

• Much less prepared

• Slightly less prepared

• About the same

• Slightly more prepared

• Much more prepared

Our Project

​

You and your partner will select a problem you see in the school to study

Choose a question

1. Determine your population

2. Select a survey method that ensures randomness

3. Write survey questions free of bias

4. Collect data

Complete a Survey

1. Analyze data

2. Propose an action based upon your information

3. Present findings to a teacher/admin panel
   

Present Findings To A Panel