Designing classroom-based tests

Designing language assessments in context

By PROF. JHON LOSADA-RIVAS

In this section, I present six fundamental qualities for language assessments. Knowing about them, although in general terms, helps to understand the assessment analysis in the last section of the paper.

​

Multiple Choice

This principle is considered as the capacity of an instrument to assess what it is supposed to assess. It implies that the interpretations made of scores in assessment should be clear and substantially justified.

• Reliability
• Accuracy
• Consistency
• Validity

Construct Validity

This is perhaps the most crucial quality of language assessments. If an assessment is not valid, it is basically useless (Fulcher, 2010). Before 1989, validity was considered as the capacity of an instrument to assess what it was supposed to assess and nothing else (Brown & Abeywickrama, 2010; Lado, 1961). After 1989, Messick’s (1989) view of validity replaced this old perspective and is now highly embraced: The interpretations that are made of scores in assessment should be clear and substantially justified; if this is the case, then there is relative present validity in score interpretations. For interpretations to be valid, naturally, assessments need to activate students’ language ability as the main construct (Bachman & Palmer, 2010).

Multiple Choice

This principle measures language skills consistently and yields clear results in scores hat accurately describe students’ language abilities. The level of consistency is key to successfully implement this principle in classroom-based assessments.

• Reliability

• Validity

• Practicality

• Authenticity

Reliability

For illustration, suppose two teachers are checking students’ final essays, so every essay receives two scores. If the scores are widely different, then there is little or no consistency in scoring, i.e. the scores are unreliable. If scores are unreliable, this will negatively impact the validity of interpretations: The two teachers are interpreting and/or assessing written productions differently.

Multiple Choice

It refers to the degree of correspondence between an assessment (its items, texts, and tasks) and the way language is used in real-life scenarios and purposes.

• Reliability
• Authenticity
• Validity
• Practicality

Authenticity

Assessments should help language teachers to evaluate how students can use the language in non-testing situations, which is why authenticity is a central quality of language assessments (Bachman & Palmer, 2010).

Multiple Choice

This principle helps students activate their language skills (i.e. the constructs of interest) and strategies for dealing with the assessment itself. It implies connecting students' topical knowledge, language skill and affective schemata.

• Reliability

• Interactiveness

• Practicality

• Washback

Interactiveness

If an assessment only stimulates a student’s knowledge of math (or any other subject) then this assessment scores low on interactiveness. Likewise, the assessment should activate the relevant topical knowledge to perform, for example, in a speaking or writing task.

Multiple Choice

Suitable and available human and material resources should contribute to the design, administration, and scoring procedure for a language assessment.

• Authenticity

• Ethics and Fairness

• Practicality

• Washback

Practicality

For instance, using a long writing assessment in a 40-student group may not be practical for scoring, as it will take too much time for one teacher to assess and interpret students’ constructs; this can be especially impractical if the teacher needs to balance other teaching responsibilities, namely planning future lessons.

Multiple Choice

It is generally considered as the impact that assessments have on teaching and learning. It can be positive or negative.

• Authenticity

• Reliability

• Washback

• Ethics and Fairness

Washback

Language assessments should lead to beneficial consequences for the stakeholders involved (Bachman & Damböck, 2018; Shohamy, 2001). In language classrooms, results of language assessments should help improve students’ language ability. Shohamy (2001) remarks one should not use them as elements of power, e.g. to discipline students for their misbehavior in class.

Multiple Choice

It refers to professional conduct to protect the assessment process from malpractice AND to the idea that all students should have the same opportunity to show their language skill.

• Authenticity

• Reliability

• Practicality

• Ethics and Fairness

Ethics and Fairness

As such, ethics and fairness are not qualities of assessment (reliability and authenticity are), but philosophical pillars that drive professional practice. Thus, ethics refers to professional conduct to protect the assessment process from malpractice; this conduct involves stakeholders in assessment, namely test-takers and professional testers (International Language Testing Association, 2000) but arguably includes language teachers and students (Arias, Maturana, & Restrepo, 2012). Fairness, on the other hand, refers to the idea that all students should have the same opportunity to show their language skills.