WorksheetsAI nondiscrimination worksheet (grade 14)
Total questions: 70
Worksheet time: 35mins
Bias persistence: Why does removing protected attributes from training data often fail to prevent discrimination?
Protected attributes are legally required
Correlated variables can still encode protected characteristics
Bias only occurs at inference time
AI models cannot process sensitive data
Data drift and discrimination: Why is keeping training data up to date important for nondiscrimination compliance?
Older data violates copyright
Societal and demographic shifts can change how risk factors impact groups over time
Regulators require monthly retraining
AI models decay automatically
Privacy-fairness tension: Why do privacy laws complicate bias testing in AI?
Bias testing is prohibited
Sensitive attributes may be restricted, limiting direct fairness measurement
Fairness overrides privacy
Only anonymised data may be used
Generative AI complexity: What makes generative AI systems harder to regulate for nondiscrimination?
They cannot be audited
Bias can emerge at multiple stages across prompts, intermediate steps and outputs
They do not use training data
They only operate in unregulated sectors
Subjective fairness: Why do technical teams often disagree on whether an AI outcome is discriminatory?
Laws define discrimination numerically
Fairness definitions depend on normative and contextual judgments
Discrimination only applies to intent
AI accuracy resolves fairness
Section 1557 trigger: Under HHS OCR Section 1557, when does an AI tool create compliance risk?
When accuracy falls below benchmark
When it produces discriminatory impacts in covered health care settings
When it is experimental
When it processes non-clinical data
Proactive duty: Section 1557 is notable because it requires covered entities to:
Respond only after complaints
Proactively identify and correct discriminatory AI impacts
Obtain pre-approval for AI
Publish source code
21st Century Cures Act relevance: Why is the 21st Century Cures Act relevant to AI nondiscrimination?
It regulates model architectures
It promotes transparency and access to health data to reduce inequities
It bans AI diagnostics
It replaces HIPAA
Health AI misconception: Which belief creates the highest compliance risk in health AI?
“AI assists clinicians”
“Clinical oversight eliminates discrimination risk”
“AI accuracy is high”
“AI improves efficiency”
Administrative AI risk: Which non-clinical AI use in health care still raises nondiscrimination concerns?
Billing prioritisation
Appointment scheduling and patient triage
Inventory forecasting
Facility maintenance
Lawful vs unlawful discrimination: What distinguishes lawful risk-based pricing from unlawful discrimination?
Whether AI is used
Whether differentiation is actuarially justified and not based on protected traits
Whether customers consent
Whether the model is explainable
AI amplification risk: Why does AI heighten discrimination risk in insurance underwriting?
AI ignores actuarial data
AI may identify proxies for protected characteristics at scale
AI eliminates regulation
AI replaces state oversight
NAIC Model Law role: The NAIC Model Law (2020) primarily serves to:
Ban AI in insurance
Guide states on regulating AI to prevent unethical or unlawful discrimination
Replace actuarial standards
Federalise insurance law
Claims processing bias: Which AI insurance function is often overlooked but high risk?
Marketing personalisation
Claims fraud detection
Website chatbots
Product naming
New York insurance guidance: New York’s AI guidance for insurers is notable because it:
Applies only to pricing
Applies to all authorised insurers using AI
Is voluntary
Replaces NAIC guidance
Core legal exposure: Why does AI screening trigger heightened legal scrutiny in employment?
Employment law bans automation
Hiring decisions affect access to work, a protected legal interest
AI lacks transparency
Models are experimental
EEOC’s key principle: The EEOC’s position on AI in hiring can be summarised as:
AI decisions are presumptively lawful
AI must meet the same nondiscrimination standards as human decisions
AI requires new employment law
AI is advisory only
Disparate impact example: Which hiring practice most clearly illustrates disparate impact risk?
Rejecting all candidates without degrees
Screening out candidates based on resume gaps correlated with caregiving responsibilities
Requiring job-specific certifications
Manual interview scoring
NYC audit requirement: NYC’s automated employment decision law requires:
Model disclosure
Independent bias audits and candidate notification
Federal approval
Consent from all applicants
Oversight fallacy: Which assumption most often leads to EEOC violations?
“We test accuracy”
“The AI only recommends”
“We audit for bias”
“We retrain regularly”
Foundational statute: Which law underpins nondiscrimination expectations in AI credit decisions?
GDPR
FCRA
HIPAA
FHA
AI-specific gap: The absence of AI-specific credit law means:
AI credit systems are unregulated
Existing financial laws must be applied to AI contexts
States regulate exclusively
Accuracy is the only requirement
CFPB signal: The CFPB’s 2021 RFI suggests the agency is:
Approving AI credit scoring
Monitoring AI practices ahead of guidance or enforcement
Delegating oversight
Ending FCRA enforcement
Explainability importance: Why is explainability critical in AI credit systems?
For public trust
Because consumers have rights to understand adverse decisions
To improve model accuracy
To reduce compute cost
Proxy risk in lending: Which variable poses the highest proxy discrimination risk?
Loan amount
Geographic indicators tied to historical segregation
Interest rate
Loan term
Foundational housing law: Which statute governs AI-based housing decisions?
FCRA
FHA
EEOC Act
NAIC Model Law
Ranking misconception: Why is “we only rank applicants” an invalid defence?
Ranking is illegal
Rankings still influence housing access outcomes
Rankings are always biased
Rankings are unregulated
HUD guidance focus: HUD’s 2020 guidance emphasises that:
AI is prohibited in housing
Automated systems must adhere to existing fair housing obligations
Only lenders are covered
Consent removes liability
Review necessity: Why must housing AI systems be regularly reviewed?
Models age quickly
Housing market dynamics and demographic effects change
HUD mandates monthly audits
AI accuracy degrades
Tenant screening risk: Which tenant screening signal is most likely to raise FHA concerns?
Prior eviction history
Criminal background proxies tied to protected groups
Income verification
Lease duration preference
Audit function: Why are bias audits central to nondiscrimination governance?
They replace regulators
They detect, document and enable remediation of discriminatory outcomes
They guarantee compliance
They eliminate human oversight
Audit timing: Which audit approach is most defensible?
One-time pre-deployment testing
Continuous or periodic audits across the system lifecycle
Only complaint-driven audits
Vendor-only audits
Vendor risk: Why do third-party AI tools increase nondiscrimination risk?
Vendors assume liability
Organisations remain responsible for outcomes even when tools are procured
Vendors hide bias
Vendors cannot be audited
Human oversight myth: Why is “human in the loop” not sufficient by itself?
Humans always agree with AI
Humans may over‑rely on AI recommendations
Laws ban automation
Oversight eliminates bias
Training effectiveness: Why is staff training part of nondiscrimination compliance?
It reduces compute cost
It helps decision‑makers understand AI limitations and bias risks
It replaces audits
It improves accuracy
Best governance response: An AI hiring tool shows lower pass rates for a protected group. What is the most defensible response?
Disable the model
Investigate data, features, thresholds and decision context
Increase accuracy
Ignore if intent was neutral
Cross‑sector pattern: Across employment, credit and housing, nondiscrimination law focuses most on:
Intent
Impact
Model size
Training time
Privacy‑safe bias testing: Which approach best balances privacy and fairness?
Avoid testing
Use protected attributes under strict governance solely for bias detection
Publish personal data
Rely on synthetic proxies only
Generative AI hiring risk: Why does generative AI introduce new hiring risks?
It replaces HR
It may embed biased language patterns into screening or evaluation outputs
It lacks accuracy
It cannot be audited
Insurance claims synthesis: An AI flags claims from a specific demographic more often for fraud review. What is the primary legal risk?
Accuracy loss
Disparate treatment in claims processing
IP infringement
Data breach
Credit decision escalation: When does AI credit scoring most likely trigger regulatory scrutiny?
When accuracy is low
When adverse decisions cannot be explained meaningfully
When models are proprietary
When models are fast
Housing ranking scenario: An AI ranks applicants but landlords decide manually. Liability risk exists because:
AI made no decision
Rankings influence outcomes and access
FHA does not apply
Consent was given
Insurance lawful discrimination boundary: Which factor most strongly legitimises differential pricing?
Business convenience
Actuarial evidence and regulatory acceptance
Model explainability
Customer consent
Governance red flag: Which is the strongest red flag for nondiscrimination risk?
High accuracy
No bias monitoring after deployment
Vendor certification
Human oversight
Multi‑law exposure: Why do AI systems often trigger multiple nondiscrimination regimes simultaneously?
AI is global
Single decisions affect multiple legally protected interests
Laws overlap accidentally
Regulators coordinate
Best documentation practice: Which documentation best supports nondiscrimination defence?
Model weights
Bias audit results and remediation actions
Marketing materials
Source code
Insurance proxy trap: Why are lifestyle variables risky in insurance AI?
They are inaccurate
They can correlate with protected traits
They reduce performance
They are illegal
Employment AI risk escalation: Which change most increases legal risk?
New UI
Expanding AI from resume screening to promotion decisions
Retraining model
Adding human review
Housing compliance defence: Which argument is weakest in defending an AI housing system?
“We regularly test for disparate impact”
“We rely on historical data”
“We corrected identified issues”
“We document decisions”
Final exam takeaway: Which statement best captures AIGP’s nondiscrimination stance on AI?
AI is neutral if data is neutral
Accuracy ensures fairness
Nondiscrimination compliance requires continuous, context‑aware governance
Laws will adapt later
Why AI hiring tools trigger nondiscrimination scrutiny: AI tools used in hiring are closely scrutinised under nondiscrimination law primarily because they:
Replace all human judgement
Influence access to employment opportunities, which is a protected legal interest
Always use sensitive personal data
Are regulated by AI-specific employment statutes
EEOC’s core position on AI in hiring: The EEOC’s 2021 guidance makes clear that AI in hiring:
Is exempt from federal nondiscrimination laws
Is permitted only if accuracy exceeds a set threshold
Must comply with existing federal nondiscrimination laws in the same way as human decision-making
Requires prior EEOC approval
Disparate impact risk: Which scenario best illustrates a disparate impact risk in AI hiring?
An AI tool rejects all candidates without degrees
An AI tool disproportionately screens out candidates from a protected group, even without explicit use of protected attributes
An AI tool is trained on synthetic data
An AI tool explains its recommendations
NYC Local Law on AI hiring: New York City’s local law on automated employment decision tools is notable because it:
Bans AI from hiring entirely
Requires public disclosure of model source code
Mandates a specific bias audit before use in employment decisions affecting NYC workers
Applies only to federal agencies
Human oversight misconception: Which statement best reflects a common compliance misconception in AI hiring?
Human review eliminates all legal risk
If a human can override AI, discrimination law does not apply
AI outputs must still be assessed for discriminatory impact even with human oversight
Oversight is required only for fully automated decisions
Foundational law for credit nondiscrimination: In the U.S., the foundational law governing nondiscrimination in credit and lending is:
The GDPR
The Fair Credit Reporting Act (FCRA)
An AI-specific federal statute
The Equal Pay Act
Lack of AI-specific statute: The absence of AI-specific credit legislation primarily means that:
AI use in credit is unregulated
Existing financial services laws must be interpreted and applied to AI systems
Only state laws apply
Credit AI systems cannot be audited
CFPB’s 2021 action: The CFPB’s 2021 request for information on AI in credit decision-making signals that the agency:
Approved all AI credit scoring
Sought to understand risks, transparency gaps and consumer harms before issuing formal rules
Delegated oversight to private auditors
Focused only on cybersecurity
Explainability requirement: Why is explainability particularly important in AI-based credit decisions?
Because models must be open source
Because consumers have rights to understand adverse credit decisions
Because explainability improves accuracy
Because regulators require model weights
Proxy discrimination risk: Which feature presents the greatest proxy discrimination risk in AI credit models?
Interest rate caps
Variables strongly correlated with protected characteristics, such as postcode or employment history patterns
Loan amount requested
Repayment schedule
Foundational housing law: The primary U.S. law governing housing discrimination remains the:
Civil Rights Act (Title VII)
Fair Housing Act (1968)
Consumer Credit Protection Act
AI Accountability Act
AI use in housing: AI systems used for tenant screening or housing decisions must:
Be certified by HUD
Demonstrate compliance with fair housing rules even when ranking or scoring applicants
Avoid using any personal data
Be fully automated
HUD’s 2020 guidance focus: HUD’s 2020 guidance primarily addressed:
Energy efficiency in housing
Automated decision-making in rental and mortgage contexts
AI explainability standards
Smart city development
Compliance misconception in housing AI: Which belief is most likely to lead to housing law violations?
“We use AI only as a recommendation tool”
“The Fair Housing Act does not apply if decisions are data-driven”
“Regular reviews reduce risk”
“AI outputs should be monitored over time”
Ongoing review importance: Why does the module emphasise regular review of housing AI systems?
Models degrade technically
Housing markets and populations change, altering impact across protected groups
Regulators require weekly audits
AI outputs are legally binding
Across hiring, credit and housing, nondiscrimination law focuses primarily on
Model transparency
Outcomes and effects on protected groups
Algorithmic novelty
Data volume
Why are bias audits particularly important in these regulated domains?
They replace regulators
They help detect, document and remediate discriminatory patterns before enforcement
They eliminate the need for human oversight
They guarantee compliance
Why do generative AI systems increase nondiscrimination risk?
They are always opaque
They involve multi-step pipelines where bias can be introduced at multiple stages
They replace all traditional models
They cannot be tested
Why do privacy laws complicate nondiscrimination compliance in AI?
Privacy laws ban bias testing
Restrictions on collecting sensitive attributes can limit the ability to detect and measure bias
Privacy laws override employment law
Privacy laws require anonymisation only
Which governance approach best aligns with the module’s guidance across all three domains?
Avoid AI entirely
Rely on vendor assurances
Combine legal review, bias audits, human oversight, and continuous monitoring
Focus only on model accuracy
