Font size
Worksheetsassesment
Total questions: 85
Worksheet time: 43mins
Fill in the blank: One of the keywords listed in the article is _________.
assessment
experiment
conclusion
variable
Fill in the blank: The article discusses the assessment of ________ competence.
professional
physical
technical
social
Based on the abstract, Miller's pyramid is used for which purpose in the context of professional competence assessment?
To assess levels of professional competence
To measure physical fitness
To evaluate financial performance
To determine patient satisfaction
According to the article, what is Miller's pyramid used for in medical education assessment?
A) Organizing review of assessment
B) Diagnosing diseases
C) Teaching patient management
D) Evaluating hospital performance
Fill in the blank: The phenomenon that candidates' performance on one case was a poor predictor of performance on any other given case, even within the same domain, is termed the '__________' problem of (clinical) competence.
content specificity
case variability
domain inconsistency
performance unpredictability
Which of the following is described as the dominant source of unreliability in Objective Structured Clinical Examination (OSCE)?
Assessor error
Content specificity
Patient error
Task variability
What is the stimulus format for the 'Shows how' layer in Miller's pyramid?
Habitual practice performance
Hands-on (patient) standardized scenario or simulation
Fact oriented
Patient scenario, simulation
What is the response format for the 'Knows' layer in Miller's pyramid?
Direct observation, checklists, rating scales, narratives
Menu, written, open, oral, computer-based
Direct observation, checklists, rating scales
Simulation
According to the passage, what is a major consequence of content specificity for assessment?
Small samples are sufficient for reliable assessment
Large samples are required to make reliable and generalisable inferences
Short tests are always generalisable
Single-point assessments are highly reliable
Fill in the blank: The wisest strategy for assessment is to combine information across content, across time and across different ________ sources.
assessment
financial
media
technical
Reliability always co-varies with the objectivity of assessment methods.
True
False
What does the acronym OSCE stand for in the context of clinical assessment procedures?
Objective Structured Clinical Examination
Organized Standardized Clinical Evaluation
Official Student Clinical Exercise
Operational Skills Competency Exam
According to the passage, what is primarily relevant to assessment situations where we cannot do without it?
Standardisation
Expert human judgement
Objective tests
Single-point assessments
What is being measured is determined more by the format of the _______ than by the format of the response.
stimulus
answer
questionnaire
response
Any assessment method is characterised by its stimulus and response formats. Fill in the blank: The stimulus is the task presented to the candidate, and the response determines how the _______ is captured.
answer
question
score
feedback
What did research reveal about validity in assessment formats?
A) Validity is determined by the response format as much as by the stimulus format.
B) Validity is determined more by the stimulus format than by the response format.
C) Validity is not affected by either format.
D) Validity is only determined by the response format.
Fill in the blank: There is evidence that written assessment formats predict _______ performance to a large extent.
OSCE
practical
oral
clinical
According to the article, what should educators worry more about when designing assessments?
Designing appropriate response formats
Designing appropriate stimulus formats
Making assessments longer
Using only multiple-choice questions
Fill in the blank: Authenticity is essential, provided the stimulus is pitched at the appropriate level of _______.
complexity
simplicity
difficulty
clarity
When a learning task is too complex, short-term memory quickly becomes overloaded and learning is hampered as a result.
True
False
According to the passage, what does good assessment crucially depend on?
The number of students
Quality assurance measures around both test development and test administration
The difficulty of the test
The length of the test
Peer review is an essential ingredient of efforts to improve the quality of test materials significantly.
True
False
Fill in the blank: Good assessment material is costly, so why not share it across schools and ______?
institutions
classrooms
teachers
departments
The idea that assessment affects learning, for better or for worse, is also termed ______ validity.
consequential
predictive
content
construct
According to the passage, what is one of the biggest wastes of capital in education?
Not sharing assessment materials across schools and institutions
Not having enough teachers
Not using technology in classrooms
Not having standardized tests
According to the article, what is a potential consequence of assessment driving learning in a certain (known) way?
It can promote negative learning effects
It can promote positive learning effects
It has no effect on learning
It only affects test scores
Fill in the blank: Single-point assessments have limitations and any form of assessment will be confined to one level of ______ pyramid.
Miller’s
Maslow’s
Bloom’s
Kirkpatrick’s
Each single assessment is a biopsy, and a series of biopsies will provide a more complete, more accurate picture.
True
False
What is required for a modern curriculum to be effective in assessment according to the article?
It should be unplanned and spontaneous
It should be planned, prepared, implemented, co-ordinated, evaluated and improved
It should only focus on test administration
It should avoid evaluations
Fill in the blank: Decisions about learners should never be based on a few assessment sources but rely on ______.
many
one
random
limited
According to the article, what does assessment at the 'does' level rely on?
Self-assessment only
Knowledgeable people to judge performance
Automated test scores
Random selection
'Expert' judgement in the context of assessment at the 'does' level refers to:
The use of experienced professionals to evaluate real-world performance.
The use of standardized tests to measure knowledge.
The reliance on peer reviews for assessment.
The use of self-assessment by the learner.
According to the passage, what is an example of the assessment of professional behaviour in tutorial groups?
School based learning environment.
Online discussion forums.
Written examinations.
Laboratory experiments.
The CanMEDS competency 'Collaborator' or 'Communicator' has wide applicability and is not unique to the medical domain.
True
False
Why is it imperative that generic skills are assessed in medical education, according to the passage?
Because these competencies are as difficult to define as their assessment is indispensable.
Because generic skills are not relevant to medical practice.
Because only technical skills matter in medical education.
Because assessment of generic skills is optional in medical training.
Which of the following is key to effective assessment at the 'does' level, as stated in the passage?
Standardized tests
Expert judgement
Peer review
Self-assessment
What are the two types of assessment instruments mentioned in the passage for judgement of performance based directly on observation or on the assessor’s exposure to the learner’s performance?
1) Judgement of performance based directly on observation or on the assessor’s exposure to the learner’s performance. 2) Aggregation instruments that compile information obtained from multiple sources over time.
1) Written examinations and oral interviews. 2) Peer review and self-assessment.
1) Standardized tests and project work. 2) Group discussions and presentations.
1) Multiple-choice questions and essay writing. 2) Laboratory experiments and fieldwork.
According to the passage, what is the typical range of observations needed for reliable inferences in direct performance measures?
2-5
8-10
15-20
25-30
The passage states that global expert judgement might bring more unique information to assessment than analytical methods.
True
False
Fill in the blank: Adequate reliability does not preclude bias in ______ judgements.
global
specific
technical
legal
One suggested measure to reduce bias in direct observation methods, according to the passage, is:
Using multiple observers
Increasing the sample size
Reducing the observation time
Focusing only on positive behaviors
According to the passage, self-assessment is considered inherently confined in validity because:
it relies on subjective judgment.
it uses standardized tests.
it is always accurate.
it is based on peer review.
According to the article, validity resides more in the users of the instruments than in the instruments that are used.
True
False
What is essential to the process of assessment and determines its value, as mentioned in the article?
Standardisation
Conscientiousness
Objectifying
Training
The article states that positive feedback has more impact than negative feedback.
True
False
Fill in the blank: The principle that at the top of the pyramid it is the ________ of effective assessment.
sine qua non
raison d'être
modus operandi
status quo
According to the article, narrative, qualitative information is often much richer and more appreciated by learners than numerical and quantitative information.
True
False
Which criterion is associated with the strategy 'Prolonged engagement'?
Credibility
Transferability
Dependability/Confirmability
None of the above
Fill in the blank: The potential assessment strategy for 'Triangulation' is to _________
Use multiple sources of assessment within or across methods.
Rely on a single assessment method for all evaluations.
Ignore conflicting data from different sources.
Focus only on quantitative data for assessment.
Fill in the blank: The criterion associated with 'Stepwise replication' is _________.
Dependability/Confirmability
Credibility
Transferability
Authenticity
Which of the following is a potential assessment strategy for 'Audit'?
Training of assessors
Organize quality assessment procedures with external auditor
Sample broadly over different contexts
Incorporate the learner's point of view
Fill in the blank: The strategy 'Member checking' incorporates the ________ in the assessment procedure.
learner's point of view
teacher's feedback
administrative guidelines
curriculum standards
According to the article, what does the 'scaffolding' of judgement rely on?
Probabilistic and empirical information.
Personal beliefs and opinions.
Historical anecdotes and myths.
Random guesses and assumptions.
What is the counterpart of reliability or generalisability in psychometric research, as mentioned in the article?
A) Aggregation
B) Saturation
C) Triangulation
D) Feedback
What is the effect of summative assessment programmes on learning, according to laboratory studies?
Positive
Negative
Neutral
No effect
What is the key to achieving deeper learning strategies, as stated in the article?
Assessment
Motivation
Technology
Collaboration
What model was recently proposed for developing assessment guidelines?
A model for developing assessment guidelines.
A model for evaluating student performance.
A model for curriculum development.
A model for teacher training.
Practice points: Assessing 'knows', 'knows how', 'shows how'. Fill in the blank: Competence is ________, not generic.
specific
universal
general
broad
Fill in the blank: Objectivity is not the same as ________.
reliability
accuracy
validity
precision
Fill in the blank: What is being measured is determined more by the ________ format than by the response format.
stimulus
answer
question
test
Practice points: Assessing 'knows', 'knows how', 'shows how'. True or False: All methods of assessment can have 'built-in' validity.
True
False
Practice points: Assessing 'knows', 'knows how', 'shows how'. Assessment drives learning.
True
False
Practice points: Assessing 'knows', 'knows how', 'shows how'. No single method can do it all.
True
False
Fill in the blank: A feasible sample is required to achieve reliable ________.
inferences
guesses
errors
opinions
Assessing 'Does'. Bias is an inherent characteristic of expert judgement.
True
False
Fill in the blank: Validity lies in the users of the instruments, more than in the ________.
instruments
methods
results
procedures
Practice points: Assessing 'Does'. True or False: Formative and summative assessment are typically combined.
True
False
Assessing 'Does'. Qualitative, narrative information carries a lot of weight.
True
False
Practice points: Assessing 'Does'. Summative decisions can be rigorous by using non/psychometric qualitative procedures.
True
False
Research agenda: Assessors reason and arrive at judgements by:
Evaluating evidence and applying criteria
Guessing based on intuition
Following a fixed checklist without analysis
Relying solely on peer opinions
Research agenda: Can we model information seeking and decision making in formalised models?
Yes, it is possible to model information seeking and decision making in formalised models.
No, it is not possible to model information seeking and decision making in formalised models.
Only information seeking can be modelled in formalised models, not decision making.
Only decision making can be modelled in formalised models, not information seeking.
Research agenda: Assessment drives learning by influencing what and how students study.
Assessment drives learning by influencing what and how students study.
Assessment has no impact on student learning.
Assessment only measures learning outcomes, not the learning process.
Assessment discourages students from studying.
Who authored the work 'Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains'?
Shavelson RJ
Ericsson KA
Norman G
Van der Vleuten CPM
Fill in the blank: The article 'Sampling variability of performance assessments' was published in the year ____.
1993
1987
2001
1998
Which reference discusses 'Pitfalls in the pursuit of objectivity: issues of reliability'?
Reference A discusses 'Pitfalls in the pursuit of objectivity: issues of reliability'.
Reference B discusses 'Pitfalls in the pursuit of objectivity: issues of reliability'.
Reference C discusses 'Pitfalls in the pursuit of objectivity: issues of reliability'.
Reference D discusses 'Pitfalls in the pursuit of objectivity: issues of reliability'.
Who are the authors of the reference: Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?
Cate O & Scheele F
Smith J & Brown L
Miller D & Johnson P
Taylor S & Evans R
In which journal was the article 'Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review' by Lurie SJ, Mooney CJ & Lyness JM published?
Acad Med
JAMA
Medical Education
The Lancet
Which year was the article 'Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review' by Kogan JR, Holmboe ES & Hauer KE published?
2009
2012
2005
2015
The article 'The mini-CEX (Clinical Evaluation Exercise): a preliminary investigation' by Norcini JJ, Blank LL, Arnold GK et al. was published in Ann Intern Med.
True
False
What is the main topic of the reference: Cruess R, McIlroy JH, Cruess S et al. 'The professionalism mini-evaluation exercise: a preliminary investigation.' Acad Med 2006 Oct; 81(10 Suppl): S74–S78.
Professionalism mini-evaluation exercise
Medical error reporting systems
Patient safety protocols
Clinical skills assessment
Who are the authors of the study titled 'Does a student log provide a means to better structure clinical education?' published in Med Educ 1999?
Dolmans D, Schmidt A, Van der Beek J et al.
Smith J, Brown L, Patel S et al.
Johnson M, Lee K, Carter P et al.
Williams R, Evans T, Green D et al.
Which publication discusses 'The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory'?
Kluger AN & DeNisi A. Psychol Bull 1996; 119: 254–284.
Bandura A. Social Foundations of Thought and Action. Prentice-Hall, 1986.
Locke EA & Latham GP. A Theory of Goal Setting and Task Performance. Prentice-Hall, 1990.
Deci EL & Ryan RM. Intrinsic Motivation and Self-Determination in Human Behavior. Plenum, 1985.
Who wrote 'The power of feedback' published in Rev Educ Res 2007?
Hattie J & Timperley H.
Bandura A & Walters R.
Black P & Wiliam D.
Dweck C & Leggett E.
What is the main topic of the book by Pendleton D, Scofield T, Tate P et al.?
The consultation: an approach to learning and teaching.
Principles of medical ethics.
Advanced pharmacology in practice.
Introduction to clinical surgery.
Which study explores how undergraduate clinical learning climates differ?
Boor K, Scheele F, Van der Vleuten CP et al. Med Educ 2008 Oct; 42(10): 1029–1036.
Harden RM, Sowden S, Dunn WR. Med Educ 1984; 18(4): 284–297.
Eva KW, Regehr G. Acad Med 2005; 80(10): S54–S57.
Norman G. Med Educ 2002; 36(8): 720–725.
