Ai and Assessments

AI and Assessments

Balance between openness and objectivity and

the avoidance of unnecessary drama

Question time:

https://www.youtube.com/watch?v=iA_nBykoAtg

—Lauren, a pupil at Wimbledon High School, a private girl’s school in south London

“I was actually surprised about how it was able to give me information that wasn’t widely available, and a different

perspective.”

“Over the past year, the school has observed that students’ assignments have become a lot
more visual. Alongside written work, students regularly submit images or videos created by

AI-powered art generators like Dall-E or Midjourney.”

—Anita Gademann

“automating young people’s day-to-day lessons by allowing AI to do the
legwork could lead to “learning loss”, a decline in essential literacy and

numeracy skills.”

--Anna Mills

AI and Assessments: the good, the bad, and the ugly

“I’m not a pessimist, but we have to collectively avoid this becoming

a dystopian thing,”

—Sir Dan Moynihan

The following is how I am measuring my assessments.

AI and Assessments

How can we ensure that our assessments objectively show that our students are
able to apply transfer skills and/or show evidence of understanding (UBD use of
term) ?

The problem: due to AI, there is a possibility that an individual could successfully
complete a course without actually gaining the targeted understanding and
transfer skill.

Chat GPT with Quillbot can do more than many teachers are
prepared for

As I have experienced, designing questions that require students to write
from their own unique perspective does not prevent them from using
chat GPT.

GPT can write from the perspective of a young Taiwanese students
(example)

Once chap GPT writes students unique personal experience from them,
students can use quillbot to hide the use of chat GPT from teachers

From the students’ perspective:

Assessments that lend themselves open to misuse of AI applications by students is
more a teacher responsibility and a student responsibility:

-Students are under pressure to achieve good grades from parents

-Students may be overwhelmed

-Students may find it difficult to avoid the temptation of AI

-Students may unwittingly overuse such AI applications

-Students might be employing a ghostwriter (a separate but related issue)

AI and Assessments

Formative (practice and participation)

Writing process assignments. Teachers can
recognize student writing styles and
teachers can remind students that they
should have an authentic attempt to write
themselves.

The grades have lower weighting, stakes are
lower, and so there is less drama. Moreover,
as this is just a formative assessment,
students could be given the opportunity to
make up the assignment.

High stakes vs. low stakes assessments,

an example

Summative (light, heavy, finals (project & test)

Teachers shouldn’t be accusing students of
misusing AI as students could falsely deny use of
AI, and teachers may falsely accuse use of AI.
Moreover, much heavier weighting of summative
assessments mean the stakes are much higher.
This kind of drama should be avoided.

Summative writing assignments are better suited
to in class timed writing.

Teachers could provide the prompt or theme
before hand. Or, the final draft of the formative
assessment (to the left) is completed via in-class
timed writing

Lower stakes

Higher stakes

A formative and summative spectrum

Writing assignment:
students can choose
their subject and they
are free to use their
computers in class
and outside of class ES
10(Argument essay)

Students design
a poster on a
preselected
topic. Students’
work will be
checked for AI.
Students will
present their
posters Ex. 1

Authenticity checks.
Used in my class and,
to my knowledge, in
science. Students had
to rewrite parts or all
of their assignment in
class as an
authenticity check. Ex.
1 (*this was not an
ideal solution)

In-class timed
writing. Students are
only given the
general themes that
they will need to
prepare an essay for,
not the writing
prompts themselves.
Ex. 1

Students are provided with
the prompt. They can freely
use Ai as a tutor to generate
ideas. There will be a
presentation and/ or
socratic discussion.
Students will have had to
acquire knowledge from
research to appropriately
complete this assignment.
Ex. 1, ex. 2

In-class
examination.
MCQ and SAQ.
Somewhat
boring.

A research essay
where students will
have to combine
information in a
somewhat unique
way. It is hard for AI
to write this kind of
assignment for
students. Ex. 1
Ex. 2

Students are
provided a prompt.
The complexity of
the prompt may
render the use of
chat GPT more
problematic. They
will answer the
prompt in video
format outside of
class. Ex. 1

Students work an assignments
together and hold each other
accountable to not use AI (or
outside help) inappropriately.
Students could also assess
each other. This is an idea
source from the web.
(personally, I am not a fan)

An assignment
(quiz or handout,
etc. )is given and
students have free
access to the web
and AI to complete
the assignment.
Students take
personal
responsibility Ex. 1

Ken’s
example:

Lower
stakes

Higher
stakes

Note: Everyone is on the spectrum

Theron’s assignment
Ai used as a tool.
Students are
encouraged to
identify and reflect
upon the significant
changes that AI
made to their
assignment. This is
part of the
assignment itself. Ex.
1

A blank template of this presentation’s formative and summative
spectrum has been sent to you all. As a group, please give a short
description of an assessment activity you have designed that
appropriately integrates AI (yes, this is somewhat subjective). Place
that activity on the provided spectrum template(G1, G2, G3, G4, G5, G6,
G7). Finally, please link that assessment. We would like to create a
resource of assignments that have taken AI into consideration. Many
thanks

Sharing Time

Bing Chat

●Bing Chat is the newest version and most capable form of ChatGPT, spurred by
Microsoft’s partnership with OpenAI (the startup that created ChatGPT). This
presentation will therefore use the terms “Bing Chat” and “ChatGPT” interchangeably.

●Whereas the original ChatGPT had major limitations, such as only having access to
information through 2021 (trained on a curated “dataset”), Bing Chat can access new
information in real-time to aid it in solving problems.

○You can actually watch Bing Chat perform multiple web searches and use idea
synthesis with basic reasoning skills to answer questions (which in turn will give
you a sense of how it works).

■Limitation: It is not great at differentiating between credible sources and less
trustworthy ones. However, it will usually include links that you can then
investigate yourself.

Bing Chat

●Bing Chat is only accessible on the Microsoft Edge Browser or with the Bing app.

○From Microsoft Edge, you can access it by going to https://www.bing.com/chat

○Important note– there are similar tools to Bing Chat being developed all the time.
Don’t think you’re safe by restricting students to only using the Chrome browser
during class time.

■That being said, Bing Chat is the most complete/reliable one, and as
mentioned before, it still has major limitations. We don’t currently have
much to worry about from competing products, as their outputs tend to be
of lower quality and therefore more easily identifiable as AI writing; however,
that will definitely change in the near future.

AI Output

●ChatGPT follows some instructions better than others. It has no real “self-awareness” of
the quality of its responses or the degree to which it has completed a task.

●It is always influenced by the person writing the prompt; the degree to which it is
influenced, however, can vary wildly.

●See the ChatGPT Argument for an example of both–

○Here, ChatGPT was unable to understand that it was repeating my own ideas back
to me, which resulted in a sort of feedback loop and eventual termination of the
chat.

●ChatGPT is also prone to “hallucinations,” which have been described in the tech media
as it being “confidently wrong” about the topic it’s addressing.

○It will not give any kinds of disclaimers about which aspects of it’s output it’s more
or less certain about, because it simply does not know.

AI as Instructional Tool

●ChatGPT can be very useful for teachers and students in certain ways.

○For example, I used the following prompt to generate relevant Cloze reading
passages, which saved me a ton of time:

■Please write a cloze reading passage containing one fill in the blank question
for each of following vocabulary words:

Ken’s Integration of AI

Ken will provide you with an assignment that

he adapted in light of AI. This assignment

both integrates AI and ensures that students

are not tempted to write the assignment

solely with AI

Day 2: Ken’s Integration of AI

Write here