Font size
WorksheetsItem analysis
Total questions: 76
Worksheet time: 2hrs 32mins
What does item analysis tells us?
Why scores on a specific test are more or less reliable or valid
The correlation of items with a factors
Looks at each item and how they interfere with the score of the test
Suggest ways to improve the validity or reliability of test score
Look at if each items interfere with the reliability and validity of your test
What does item analysis tells us?
Item analysis is done on top of/in addition to what else?
Reliability
Utility
Factors analysis
Validity
Which item characteristics are only relevant for maximal performance test?
Item discrimination
Item analysis
Item difficulty
Distractor power
Item-total correlation
Which item characteristics are for tests with right and wrong answers?
Item discrimination
Item analysis
Item difficulty
Distractor power
Item-total correlation
When looking at item difficulty what is important to have as part of your test?
Items related to the construct
Difficult items
Easy items
Items correlated to the construct
None of the above
When looking at item difficulty what is important to have as part of your test? (Explain why)
How is item difficulty assessed?
Using item difficulty correlation
Using item difficulty variance
Using item difficulty index
Using item total correlation
What symbol denotes item difficulty index?
(a)
Which of the following relates to the difficulty index?
Varies between -1 and 1
Varies between 0 and 1
The ideal is to have an average p of 0.5
The ideal is to have an average p of 0.3
What is the ideal average difficulty index?
(a)
Interpreted the following p value for a difficulty index.
Bob is looking at the results of his test. He finds that one of his items have a p of 0.5. What does that mean?
(a)
Interpreted the following p value for a difficulty index.
Bob is looking at the results of his test. He finds that one of his items have a p of 1. What does that mean?
(a)
Interpreted the following p value for a difficulty index.
Bob is looking at the results of his test. He finds that one of his items have a p of 0. What does that mean?
(a)
Bob has a problem. In on of his tests he has gotten an item that has a p value (item difficulty index) of 0. What should he do? (explain your reasoning)
(a)
True or False
A p value of 0.5 in itself is all that needs to be looked at when looking at item difficulty.
True
False
True or False
A p value of 0.5 in itself is all that needs to be looked at when looking at item difficulty. (explain your answer)
When looking at the value for item difficulty what do we need to keep in mind?
(a)
Alia, a student working in her teachers lab, is asked to calculate the p value for a test. To help her with her work, her teacher gives her an example to follow. Sadly some of the information of the example is messing, but the valuable information are still present. Help Alia by finding the p value.
Number of participants in the test is 60. It is a multiple choice item with 4 choices.
p = 0.5
p = 0.625
p = 0
It is impossible to find the P (not enough information)
For a multiple choice question, what is the ideal p level?
50% of the remaining interval after taking guessing into account
50% of the interval before taking guessing into account
50% of the interval which is always the same
None of the above
In the following picture, what constitute an ideal p value?
A
B
C
D
E
Which of the following is also called item validity?
Item difficulty
Item difficult index
Item discrimination
Distractor power
What is the following describing?
Item discrimination analysis assesses the degree to which different types of persons respond differently to the items.
Item difficulty
Item difficult index
Item discrimination
Distractor power
Which of the following is the definition of item discrimination?
The extent to which the item is reliable
The extent to which the items measure the same thing as the test
The extent to which the test measures the thing it is supposed to measure
Which item characteristics are relevant for maximal performance and typical test?
Item discrimination
Item analysis
Item difficulty
Distractor power
Item-total correlation
What is item discrimination?
When it comes to item discrimination, for maximal test good items are correctly answered by the strongest respondents and incorrectly answered by the weaker respondents. This is assessed using ___________.
Item difficulty index
Item-total correlation
Item discrimination index
Correlation
Regression
When it comes to item discrimination, for typical test good items are not answered in the same manner for respondents who possesses the characteristics versus those who don't presents the characteristic. This is assessed using ___________.
Item difficulty index
Item-total correlation
Item discrimination index
Correlation
Regression
For item discrimination, for maximal performance test when are items considered good?
(a)
For item discrimination, for typical performance test when are items considered good?
(a)
In item discrimination for__________ performance test items are considered good when strongest respondents respond correctly while weakest respondents answer incorrectly. That is done using __________.
Maximal
Typical
Content validity
Item discrimination index
Item total correlation
In item discrimination for__________ performance test items are considered good when those who possesses the characteristic responds differently from those who do not posses the characteristic. That is done using __________.
Maximal
Typical
Content validity
Item discrimination index
Item total correlation
What is the difference between maximal performance test and typical performance test when it comes to item discrimination?
What symbol denotes item discrimination index?
P
D
PB
q
PT
How id item discrimination calculated? (the formula)
(a)
What is the correct order of calculating the item discrimination index?
1)calculate the proportion of top scorer with right answer
2)Calculate the item discrimination index
3)Divide group into strongest and weakest
4)Calculate the proportion of bottom scorer with right answer
1, 2, 3, 4
2, 3, 5, 4
3, 1, 4, 2
4, 1, 3, 2
An ideal discrimination for item discrimination is _______.
D = 0.5
D = 1
D = 0
D = 0.3
Bob is looking back at a test he administered earlier to his students. Explain what each item means for Bobs tests/items.
In item #1 d = -1
In item #2 d = 0.3
In item #3 d = 0.5
In Item #4 d = 1
In Item #5 d = 0
What item analysis characteristics need to be higher than 0.3?
Item difficulty index
Item discrimination index
Item total correlation
Distractor power
Alia for one of her school assignment has to find the item discrimination index of a true/false typical test, but she has no idea on how that would even work. She asks you for help. What should you tell her regarding typical test and item discrimination to help her?
What is an ideal item total correlation?
(a)
What is an ideal discrimination index?
(a)
What is excluded when calculating the item total correlation? (Explain why)
What is the expected distractor power of the items indicated by A, B, and C?
(a)
Which of the following indicates problematic distractor power?
Items level difficulty unclear
Distractor too weak
Distractor too obviously erroneous
Distractor too plausible
Which problematic type of distractor does the following statement describes?
Increases the possibility of success based on guessing.
Items level difficulty unclear
Distractor too weak
Distractor too obviously erroneous
Distractor too plausible
Which problematic type of distractor does the following statement describes?
Item will perform the same way as if it didn't exist.
Items level difficulty unclear
Distractor too weak
Distractor too obviously erroneous
Distractor too plausible
Which problematic type of distractor does the following statement describes?
Will cause people who are knowledgeable to choose that particular item
Items level difficulty unclear
Distractor too weak
Distractor too obviously erroneous
Distractor too plausible
What are the two possible problematic distractor types and explain them.
What assumption does distractor power relies on?
(a)
Bob is administering a test for his class of 19 students, and want to know what the expected distractor power is for one of his items. For this item there are 4 choice response choices and 9 students did not get the right answer. What is the expected distractor power of this item?
(a)
The common assumption associated with distractor power is associated with which of the following?
Process
Actual Distractor power
Distractor power
Expected distractor power
What do we call items that are never selected?
Process
Actual Distractor power
Distractor power
Expected distractor power
What do we call items that are selected more than they should?
Process
Actual Distractor power
Distractor power
Expected distractor power
How do we call items that are never selected (for distractor power)?
(a)
How do we call items that are selected more than they should (for distractor power)?
(a)
What does "Process" means in the context of distractor power?
(a)
Give the definition of actual distractor power.
(a)
What can be done/looked at to have a greater amount of precision for distractor power?
(a)
What characteristic of item analysis is used to interpret the distractor power?
Item difficulty
Item discrimination
Process
None of the above
Variance
What characteristic of item analysis is used to interpret the distractor power?
(a)
What other item characteristics helps interpret distractor power? And what/how does it help us?
How are all the item characteristics interrelated?
Compare and contrast the different type of item analysis characteristic.
What does high item-total correlation indicates?
(a)
Bob used an item total correlation for one of its test. One of his item has a low item-total correlation.
1) What type of performance test is Bob is analyzing?
2) What does his result (low item-total correlation) tells us?
True or False
An item discrimination power of -1 for a typical test (using the same analysis of a maximal test and a typical test) is problematic.
True
False
In what case is a d= 1 is perfect discrimination?
When using discrimination index for typical test
When using discrimination index for maximal test
Never
Always
In what case is a d= -1 is problematic?
When using discrimination index for typical test
When using discrimination index for maximal test
Never
Always
In what case is a d= -1 is not problematic?
When using discrimination index for typical test
When using discrimination index for maximal test
Never
Always
Alia is using a discrimination index for a typical test with true/false answer. She is panicking because her calculation shows that a particular item has a d =-1. What should you tell her?
Why is item analysis significant/important?
What does item analysis assesses?
(a)
(a) assesses the degree to which different people response differently to an item.
What is the ideal procedure for typical tests with a continuous response scale and sometime binary (true-false)?
(a)
What is item total ideal for?
(a)
