The extent to which a score from a test or from an evaluation is

consistent and free from error.

If a score from a measure is not stable or error-free, it is not useful.

If applicants score differently each time they take a test, we are unsure

of their actual scores. Consequently, the scores from the selection

measure are of big value.

Test reliability is determined in four ways

The extent to which repeated administration of

the same test will achieve similar results.

the consistency of test

scores across time.

Test retest: There is no standard amount of time that should

elapse between the two administrations of the

test. However, the time interval should be long

enough so that the specific test answers have

not been memorized, but short enough so that

the person has not changed significantly.

Test-Retest Reliability: Test-retest reliability is appropriate for all kinds of tests. It would not make sense to

measure the test-retest reliability of a test

designed to measure short-term moods or

feelings.

The extent to which two forms of the same test are

similar.

a method of controlling for

order effects by giving half of a sample Test A first,

followed by Test B, and giving the other half of the

sample Test B first, followed by Test A.

the extent to which the scores on

two forms of a test are similar.

Looking at the consistency with which an

applicant responds to items measuring a

similar dimension or construct

the extent to which responses

to the same test are consistent.

another factor that can

affect the internal reliability whereas it is the

extent to which test items measure the same

construct.

When reading information about internal

consistency in a journal article or a test

manual, you will encounter three terms that

refer to the method used to determine internal

consistency

a form of internal reliability in which the consistency of item responses is determined by comparing scores on half of the items with scores on the

other half of the items.

corrects reliability coefficients

resulting from the split-half method.

a statistic used to determine internal reliability of tests that use interval or ratio scales.

the extent to which two people scoring a test

agree on the test score, or the extent to which

a test is scored correctly.

It is especially an issue in projective or

subjective tests in which there is no one

correct answer, but even tests scored with the

use of keys suffer from scorer mistakes.

When human judgment of performance is

involved, scorer reliability is discussed in

terms of interrater reliability. That is, will two

interviewers give an applicant similar ratings,

or will two supervisors give an employee

similar performance ratings?

The degree to which inferences from test scores are justified by the evidence.

Even though reliability and validity are not the same, they are related. The potential validity of a test is limited by its reliability. Thus, if a test has poor reliability, it cannot have high validity.

Instead, we think of reliability as having a necessary but not sufficient

relationship with validity.

The extent to which tests or test items sample the content that

they are supposed to measure.

In industry, the appropriate content for a test or test battery is

determined by the letter of recommendation.

One way to test the content validity of a test is to have _____ ____ _____ (e.g., experienced employees, supervisors) rate

test items on the extent to which the content and level of

difficulty for each item are related to the job in question.

The readability of a test is a good example of how tricky

content validity can be.

The extent to which a test score is related to some measure of job performance.

a measure of job performance, such as attendance,

productivity, or a supervisor rating.

Criterion validity is established using one of three research designs:

a form of criterion validity that

correlates test scores with measures of job performance

for employees currently working for an organization.

a form of criterion validity in which

test scores of applicants are compared at a later date with

a measure of job performance.

a narrow range of performance scores that

makes it difficult to obtain a significant validity coefficient.

a major issue concerning the

criterion validity of tests whereas it is the extent to which

inferences from test scores from one organization can be applied

to another organization.

a form of validity generalization in which

validity is inferred on the basis of a match between job

components and tests previously found valid for those job

components.

is the most theoretical of the validity types. The extent to which

a test actually measures the construct that it purports to

measure.

a form of validity in which test scores

from two contrasting groups “known” to differ on a construct are

compared.

To get a significant validity coefficient, many things have to go

right. You need a good test, a good measure of performance,

and a decent sample size.

Next-door neighbor rule is advisable to use to check if content validity is enough

Finally, a test itself can be valid. When we speak of

validity, we are speaking about the validity of the test scores as

they relate to a particular job.

The extent to which a test appears to be valid.

This perception is important because if a test or its items do not

appear valid, the test takers and administrators will not have

confidence in the results.

If job applicants do not think a test is job related, their

perceptions of its fairness decrease, as does their motivation to

do well on the test (

If two or more tests have similar validities, then cost should be considered. A particular test is usually designed to be administered either to individual

applicants or to a group of applicants. Certainly, group testing is usually

less expensive and more efficient than individual testing, although

important information may be lost in group testing.

a type of test taken on a computer in which the computer

adapts the difficulty level of questions asked to the test taker’s success in answering previous questions.

The advantages to CAT is that fewer test items are required, tests take less

time to complete, finer distinctions in applicant ability can be made, test

takers can receive immediate feedback, and test scores can be interpreted

not only on the number of questions answered correctly, but on which

questions were correctly answered.

A set of tables based

on the selection ratio, base rate, and test

validity that reveal the percentage of future

employees who will be successful if a

specific test is employed.

A utility

approach that compares the number of

times a selection decision was correct with

the number of successful employees.

creates tables

Use the base rate, test validity, and

as well as applicant percentile on a

test to determine the likelihood

in terms of future success for that

applicant.

Method of utility formula

determining the degree to which

a company will gain from

the usage of a specific selection

system

It is simply the number of people hired

for a specific role in a company.

This is the average amount of time that employees in the spend on the job. Positions tend to be retained by the organization.

This is the criterion validity coefficient that was calculated.

achieved through either a validity research or validity generalization.

o determine this, add the total salary

of current employees in the position in question.

The answers to the questions should be averaged.

There are two ways

to get this number. The first approach is to calculate the average score. On the selection

test for both the hired applicants and the applicants who are not employed.

The second way is to compute the proportion of applicants who are successful.

employed, followed by a conversion

refers to technical aspects of a test.

group differences (e.g., sex, race, or age) in test

scores that are unrelated to the construct being

measured.

relationship between a test and an outside criterion.

can be tested through regression analysis and is

deemed present if there is a difference in slope or

intercept of the subgroup.

rank-orders applicants on the basis of their test scores.

Selection is then made by starting with the highest score and

moving down until all openings have been filled.

to top-down selection, the

assumption is that if multiple test scores are used, the

relationship between a low score on one test can be

compensated for by a high score on another.

are a means for reducing adverse impact and increasing

flexibility. With this system, an organization determines the

lowest score on a test that is associated with acceptable

performance on the job.

Both of these approaches are

used when one score can’t compensate for another or when

the relationship between the selection test and performance is

not linear.

is a model in which the applicants would be

administered all of the tests at one time. If they failed any of the tests (fell below the passing score), they would not be considered further for

employment.

attempts to hire the top test scorers while still allowing some

flexibility for affirmative action

Banding

takes into consideration the degree of error associated with any

test score. Thus, even though one applicant might score two

points higher than another, the two-point difference might be the

result of chance (error) rather than actual differences in ability.