Phishing & AI Training Fall 2025

​Phishing & AI Privacy

By Megan Hull-Burg

Fall 2025

Open Ended

How does the exposure of student information like bus routes, medical records, and Social Security numbers change the way we think about school safety?

Poll

Which tactic do you think is most convincing to staff in schools?

• attachments

• urgent requests

• fake branding

AI Privacy Issues for teachers

• Data Collection

• Data Storage

• FERPA/COPPA

• Bias/Fairness

Outline of topics

Teacher's Guide to AI Privacy Issues

• Transparency

• Long Term Risks

• Safe Practices

​AI tools may collect metadata (IP addresses, typing patterns, usage times) even if you don’t enter names.

This data can still be used to profile or track students.

Teachers should ask: What data is collected, stored, and shared by this tool?

Data Collection

Under FERPA, schools must protect student educational records.

Under COPPA, parental consent is required for collecting personal data from children under 13.

Many AI tools aren’t designed with these laws in mind.

Data Storage

AI tools may store inputs for future training or quality control.

Teachers should check: Who owns the data once it’s in the system: the teacher, the student, or the company?

FERPA/COPPA

​AI models may reflect biases from the data they were trained on.

If teachers use AI for feedback or grading support, it could unintentionally disadvantage certain groups of students.

Bias/Fairness

Data uploaded today could persist for years. Even if anonymized, it might be combined with other data later.

Teachers should emphasize the digital footprint students leave when using AI tools.

Transparency

Many AI tools don’t explain how they generate responses.

Teachers should be cautious about relying on "black box" systems for decisions that impact students.

Long Term Risks

• DSD AI School Board Policy is coming soon!

  • Be sure to read and understand this document when it is shared!

• Check the Privacy Policy of tools before using them. Look for what data is stored, sold, or shared and with whom!

​

​Safe Practices

• Never enter student Personally Identifiable Information (PII) into a large language model like ChatGPT or an AI tool like MagicSchoolai.

• PII includes anything that could be used to identify a student like:

  • Full name

  • Home Address

  • Date of Birth

  • etc.

• Keep student information generalized (e.g., “a 7th grader struggling with fractions” instead of “Alex Smith in 7th grade at Donegal”).

​

Multiple Choice

You find a new AI tool advertised as “perfect for classrooms.” What’s the best next step before using it?

• Test it right away with student work.

• Share student data to see how it performs.

• Read the Privacy Policy and request district approval.

• Ask students to create accounts without parental notice.

Multiple Choice

Which of the following is NOT considered Personally Identifiable Information (PII)?

• A student’s full name

• A student’s home address

• A student’s date of birth

• A 7th grader struggling with fractions

Multiple Choice

When checking the Privacy Policy of an AI tool, which detail would raise the biggest concern?

• The tool allows teachers to customize practice problems

• The company states it sells data to third-party advertisers

• The platform provides a free teacher account

• The tool’s interface is colorful and easy for students to use

Multiple Choice

A teacher wants to upload writing samples into an AI grader. What is the safest practice?

• Upload only samples with student names removed or replaced with generic labels (e.g., “Student A”).

• Upload all student essays as they are, since the AI tool is secure.

• Upload only the essays of students who scored poorly.

• Skip reviewing the Privacy Policy since the tool is “for education.”

Multiple Choice

Why should teachers avoid relying solely on AI for grading?

• AI may not grade fast enough.

• Parents may not like technology use.

• Students may prefer paper feedback.

• AI might reflect biases and cannot explain its reasoning.