Train the Bot: How Bias Gets Baked In

When you picture a "good leader," which of these comes to mind first?

• A politician giving a speech

• A coach motivating a team

• A CEO running a company

• A teacher guiding students

• A community organizer

Tags

CCSS.RL.2.6

CCSS.RL.8.3

You've probably used AI tools before — like a recommendation algorithm or a chatbot. Where does an AI get its "knowledge" from?

• It is programmed with every possible right answer

• It learns patterns from large collections of examples called training data

• It searches the internet every time someone asks a question

• It copies answers from a textbook database

What is the main reason AI systems become biased?

• AI computers are designed to be mean

• AI learns from data that may have unfair patterns

• AI doesn't have enough memory

• AI is too expensive to build correctly

Answer explanation

AI learns patterns from data — and if the data has unfair patterns, the AI copies them. It's not intentional meanness, just learning from imperfect information.

In Joy Buolamwini's facial recognition study, which group had the WORST accuracy?

• Light-skinned men

• Light-skinned women

• Dark-skinned men

• Dark-skinned women

Answer explanation

Dark-skinned women had a 34.7% error rate — compared to just 0.8% for light-skinned men. That's because the AI trained mostly on lighter-skinned faces.

Tags

CCSS.RI.6.7

CCSS.RI.7.7

CCSS.RL.6.7

CCSS.RL.7.7

CCSS.RL.8.7

Why did the healthcare AI give less help to Black patients?

• It was programmed to ignore requests from doctors

• Black patients were actually healthier

• The AI couldn't read certain types of records

• The AI used money spent on healthcare to predict who needed help

Answer explanation

The AI used money spent on healthcare to predict who needed help. But Black patients had historically spent less — not because they were healthier, but because they had less access to care. The AI misread the data.

Which of these is a good way to reduce AI bias?

• Only use AI for unimportant decisions

• Use training data from just one well-studied group

• Test AI on all different groups before releasing it

• Make AI decisions secret so no one complains

Answer explanation

Testing on all groups helps catch bias before it hurts real people. Transparency, diverse data, and diverse teams are all part of the solution.

Should AI be used to make important decisions about people — like who gets hired for a job or who gets medical help? Why or why not?

Let's check your knowledge! Match each term to its definition.

• Bias

• The result or answer an AI produces

• Representation

• When an AI's outputs unfairly favor certain groups over others

• Output

• Whether different types of people or perspectives are included in the data

• Training data

• The examples used to teach an AI a concept

You are about to act as an AI trainer. Your job is to choose example sentences that will teach a fictional AI what a "good leader" looks like. Before you start — in your own words, what do YOU think makes someone a good leader? Write 1–2 sentences.

Tags

CCSS.RL.2.6

CCSS.RL.8.3

You are now training the bot. Select the sentences you want to feed the AI to teach it what a "good leader" is. Choose as many as feel right to you.

• "The CEO led his company to record profits last year."

• "She organized her neighbors to fight for cleaner water in their community."

• "The general commanded thousands of troops in battle."

• "He coached a youth soccer team and helped every player improve."

• "The president signed the new law after months of negotiation."

Tags

CCSS.RL.2.6

CCSS.RL.8.3