Item analysis

What does item analysis tells us?

• Why scores on a specific test are more or less reliable or valid

• The correlation of items with a factors

• Looks at each item and how they interfere with the score of the test

• Suggest ways to improve the validity or reliability of test score

• Look at if each items interfere with the reliability and validity of your test

What does item analysis tells us?

Evaluate responses using AI:

OFF

Answer explanation

Possible answer: Tells us why scores on a specific test are more or less reliable or valid, and suggests ways to improve the reliability or validity of these scores.

Possible answer: Looks at if each item is interfering with validity or reliability of test score. It locates items that are problematic

Item analysis is done on top of/in addition to what else?

• Reliability

• Utility

• Factors analysis

• Validity

Which item characteristics are only relevant for maximal performance test?

• Item discrimination

• Item analysis

• Item difficulty

• Distractor power

• Item-total correlation

Which item characteristics are for tests with right and wrong answers?

• Item discrimination

• Item analysis

• Item difficulty

• Distractor power

• Item-total correlation

When looking at item difficulty what is important to have as part of your test?

• Items related to the construct

• Difficult items

• Easy items

• Items correlated to the construct

• None of the above

When looking at item difficulty what is important to have as part of your test? (Explain why)

Evaluate responses using AI:

OFF

Answer explanation

Having items varying in levels of difficulty important.

Reason #1: Because it helps establish a comparison between strong and weak scorers. If all items are similar in difficulty than we no longer have variability but a constant.

Reason #2: Because it helps put the participants in confidence and avoids inflating anxiety.

Reason #3: Items that are too easy, where everyone gets the answer and items that are too hard where no one gets the answer is useless. Those do not give any sort of variability and become once again a constant.

Reason #4: With an increasing level of difficulty we get a normal curve distribution. Which helps differentiate between participants

How is item difficulty assessed?

• Using item difficulty correlation

• Using item difficulty variance

• Using item difficulty index

• Using item total correlation

What symbol denotes item difficulty index?

Which of the following relates to the difficulty index?

• Varies between -1 and 1

• Varies between 0 and 1

• The ideal is to have an average p of 0.5

• The ideal is to have an average p of 0.3