3. Q2_M1_LESSON 3_Research Instrument, Validity and Reliability

Research Instrument,

Validity and Reliability

By Edgar Monte

What I Need to Know

Replace this with your body text. Duplicate this text as many times as you would like. All provided templates can be reused multiple times. Wish you a good day.

Happy teaching!

What I Need to Know

Quantitative Research Instrument

What do you think will happen if tools for building a house is not prepared meticulously? The same thing when getting information for answers to a research problem, tools, or instruments should be prepared carefully. In constructing a quantitative research instrument, it is very important to remember that the tools created should require responses or data that will be numerically analyzed.

What I Need to Know

Research Instruments are basic tools researchers used to gather data for specific research problems. Common instruments are performance tests, questionnaires, interviews, and observation checklist. The first two instruments are usually used in quantitative research, while the last two instruments are often in qualitative research. However, interviews and observation checklists can still be used in quantitative research once the information gathered is translated into numerical data.

What’s New

Activity 1. Let’s talk

Directions: Read the questions about tools. Then, write your answer on the corresponding space.

Open Ended

Why is it necessary to use the right tools for a particular task?

Open Ended

What will happen if research instruments are not prepared carefully?

Activity 2. Let’s read

Directions: Read and analyze the given scenario. Answer the guide questions below.

Open Ended

What do you think must have been done to avoid the said situation?

Open Ended

What can you say about the result of investigation?

Open Ended

How will you relate the scenario to the conduct of a quantitative research study?

What Is It

In constructing the research instrument of the study, there are many factors to be considered. The type of instrument, reasons for choosing the type, and the description and conceptual definition of its parts are some of the factors that need to be decided before constructing a research instrument. Furthermore, it is also very important to understand the concepts of scales of research instruments and how to establish validity and reliability of instruments.

Characteristics of a Good Research Instrument

Concise

Have you tried answering a very long test, and because of its length, you just pick the answer without even reading it? A good research instrument is concise in length yet can elicit the needed data.

Sequential

Questions or items must be arranged well. It is recommended to arrange it from simplest to the most complex. In this way, the instrument will be more favorable to the respondents to answer.

Valid and reliable

The instrument should pass the tests of validity and reliability to get more appropriate and accurate information.

Easily tabulated.

Since you will be constructing an instrument for quantitative research, this factor should be considered. Hence, before crafting the instruments, the researcher makes sure that the variable and research questions are established. These will be an important basis for making items in the research instruments.

Ways in Developing Research Instrument

There are three ways you can consider in developing the research instrument for your study. First is adopting an instrument from the already utilized instruments from previous related studies. The second way is modifying an existing instrument when the available instruments do not yield the exact data that will answer the research problem. And the third way is when the researcher made his own instrument that corresponds to the variable and scope of his current study.

Types of Validity of Instrument

Validity

A research instrument is considered valid if it measures what it supposed to measure.
When measuring oral communication proficiency level of students, speech performance using rubric, or rating scale is more valid than students are given multiple choice tests.
Validity also has several types: face, content, construct, concurrent, and predictive validity.

Face Validity

It is also known as “logical validity.” It calls for an initiative judgment of the instruments as it “appear.” Just by looking at the instrument, the researcher decides if it is valid.

Content Validity

An instrument that is judged with content validity meets the objectives of the study. It is done by checking the statements or questions if this elicits the needed information. Experts in the field of interest can also provide specific elements that should be measured by the instrument.

Construct Validity

It refers to the validity of instruments as it corresponds to the theoretical construct of the study. It is concerning if a specific measure relates to other measures.

Predictive Validity

When the instrument can produce results similar to those similar tests that will be employed in the future, it has predictive validity. This is particularly useful for the aptitude test.

Reliability of Instrument

Reliability refers to the consistency of the measures or results of the instrument.

Test-retest Reliability

It is achieved by giving the same test to the same group of respondents twice. The consistency of the two scores will be checked.

Equivalent Forms Reliability

It is established by administering two identical tests except for wordings to the same group of respondents.

Internal Consistency Reliability

It determines how well the items measure the same construct. It is reasonable that when a respondent gets a high score in one item, he will also get one in similar items. There are three ways to measure the internal consistency; through the split-half coefficient, Cronbach’s alpha, and Kuder-Richardson formula.

Open Ended

Activity 4. Search and evaluate the instrument.

Directions: Search for a sample research study. Identify the ways of establishing the validity and reliability of the instrument.

Activity 5. Let’s go online.

Practice what you have learned from this lesson. Visit the following:

https://bit.ly/2A7Q6Iz

https://bit.ly/2ZD2Kda

https://bit.ly/2B0cbcd

Multiple Choice

Which of the following statements most likely involves using data in a way that is not valid?

• University data indicate that the highest paid professors have the most publications.

• Students who can type more than 80 words per minute will receive a typing award.

• All of the teachers interviewed described the principal as "outstanding."

• A teacher uses self evaluations to grade writing ability

Multiple Choice

Which of the following is content-related evidence of validity?

• 
  Test items are at an appropriate reading level

• 
  Observed participant characteristics are consistent with responses on a scale.

• Scores obtained on two administrations of the instrument are consistent.

• Scores are correlated with scores obtained on another instrument.

Multiple Choice

The best construct-related evidence of validity comes from

• a series of studies of the instrument.

• a validity coefficient that is near 0.

• careful review of the instrument by experts.

• data-based predictions that prove to be correct.

Multiple Choice

Which of the following statements accurately portrays the relationship of reliability to validity?

• Inferences must be valid before the scores can be reliable.

• Scores must be reliable before inferences can be valid.

• The more valid the inference is, the higher the reliability of the score must be.

• Score reliability is not related to inference validity.

Multiple Choice

Which of the following is most likely to be the alternate forms reliability coefficient for a well-established standardized test?

• -0.93

• 
  +0.46

• 
  +0.89

• 
  +1.42

Multiple Choice

Measurement error cannot be assessed by

• giving the instrument to the same group twice

• comparing scores on different parts of the instrument

• 
  giving the instrument to two different groups

• giving two forms of the instrument to the same group

Multiple Choice

What numbers put in place of "A," "B," and "C" would provide the best criterion-related evidence of validity?

• A = 62, B = 2, C = 8

• A = 71, B = 38, C = 49

• A = 4, B = 3, C = 8

• 
  A = 1, B = 6, C = 51

Multiple Choice

What are the two forms of criterion-related validity?

• 
  correlational and content

• predictive and correlational

• predictive and concurrent

• 
  criterion and predictive

Multiple Choice

To obtain evidence of predictive validity a researcher will have to

• consult another researcher.

• 
  review all of the instruments.

• gather instrument data and criterion data at nearly the same time.

• allow a time interval to elapse between administration of the instrument and obtaining the criterion scores.

Multiple Choice

A researcher administers a history aptitude test to a group of middle school students and later compares their test scores with their end-of-the-semester grade in history. This is an example of obtaining evidence of

• 
  predictive validity

• concurrent validity

• content related evidence of validity

Multiple Choice

Which of the following is not one of the three best-known ways to obtain a reliability coefficient?

• test-retest

• equivalent forms

• 
  internal consistency

• concurrent

Multiple Choice

Which of the following are the three best known ways to obtain a reliability coefficient?

• predictive, concurrent, and equivalent forms

• test-retest, equivalent forms, and internal consistency

• test-retest, concurrent, and equivalent forms

Multiple Choice

To obtain evidence of concurrent validity a researcher will have to

• consult another researcher.

• review all of the instruments.

• gather instrument data and criterion data at nearly the same time.

• allow a time interval to elapse between administration of the instrument and obtaining the criterion scores.

Multiple Choice

If a researcher wishes to obtain content-related evidence of validity, a common way to do this is to

• 
  have a friend look over the instrument and give feedback.

• obtain a Kuder-Richardson analysis.

• 
  review the instrument very carefully.

• have an individual who knows enough about what is to be measured review the content and format of the instrument and judge whether it is appropriate.

Multiple Choice

What are the three main types of evidence on validity that a researcher might collect?

• 
  content-related, criterion-related on, construct-related

• 
  historical, criterion-related

• content-related, laboratory, construct-related

• content-related, criterion-related, observation-related

  
  

True/False

Multiple Choice

Criterion-related evidence of validity requires a second instrument.

• TRUE

• FALSE

Multiple Choice

Content-related evidence of validity refers to the degree to which information provided by an instrument agrees with information obtained on other, independent instruments.

• TRUE

• FALSE

Multiple Choice

Scoring agreement does not require a demonstration that independent scorers can achieve satisfactory agreement in their scoring.

• TRUE

• FALSE

Multiple Choice

Content-related evidence of validity includes content in the test that is consistent with what the researcher hopes to measure.

• TRUE

• FALSE

Multiple Choice

The use of expert opinion, observations, criterion checks, and other methods should be used to establish that the score of an instrument accurately reflects where the individual should be placed on a scale of the construct.

• TRUE

• FALSE

Multiple Choice

If an instrument produces varying results for the same individual, then there is no way to determine what score is the best indicator of the characteristics of the individual.

• TRUE

• FALSE

Multiple Choice

Content-related evidence of validity refers to the degree to which the totality of evidence obtained is consistent with theoretical expectations.

• TRUE

• FALSE

Multiple Choice

The test-retest method is used to estimate consistency without regard to time.

• TRUE

• FALSE

Multiple Choice

Errors of measurement refer to variations in scores obtained by the same individuals on the same instrument.

• TRUE

• FALSE

Multiple Choice

When instrument scores can be used to predict scores on a criterion measure that is collected at a later time, these scores have predictive validity.

• TRUE

• FALSE

Multiple Choice Quiz

Multiple Choice

Surveys can be:

• self-administered in writing without a researcher present.

• 
  administered by an interviewer over the phone.

• 
  administered by an interviewer in person.

• completed over the Web or email.

• all of the above.

Multiple Choice

A social desirability response is:

• 
  the correct answer to the question.

• 
  the answer the participants want to give, but feels uncomfortable sharing.

• 
  the answer the participant believes the interviewer or researcher is seeking.

• 
  the answer the interviewer indicates he or she desires.

• a response given by the participant indicating that he or she declines to answer.

Multiple Choice

Using existing questionnaires and surveys:

• 
  is preferred because these questionnaires and surveys have undergone extensive testing and refinement.

• 
  is preferred because researchers are paid each time you use their questionnaire or survey.

• is not necessary because any competent person can develop a questionnaire better suited to their research needs.

• 
  is not possible because many questionnaires and surveys are copyrighted.

• 
  is not practical because you may need to make minor modifications.

Multiple Choice

If you must write your own questionnaire or survey, a good survey item is one that:

• 
  uses difficult terminology.

• is vague or abstract.

• 
  is several sentences long.

• is straightforward.

• 
  has a socially desirable answer.

Multiple Choice

Closed questions are most effectively used when:

• 
  the question asks for sensitive information.

• 
  a Likert-type scale can be used.

• all potential responses are known in advance.

• the question asks about something that is relatively new or unique.

• multiple responses are required for the participant to fully answer the question.

Multiple Choice

Likert-type scales:

• are based on two bipolar adjectives.

• assign numerical values to each response choice.

• 
  are ratio level data.

• have 4 or 6 response choices.

• are open questions.

Multiple Choice

An open question:

• 
  provides data from the researcher's point of view.

• provides data from the respondent's point of view.

• creates easily comparable responses.

• 
  is always preferred over a closed question.

• should be used when the set of response choices is known.

Multiple Choice

Response rate is the:

• size of the population.

• size of the sample.

• return rate.

• internal reliability.

• number of questionnaires each research assistant must get filled out.

Multiple Choice

Internal reliability is the degree to which:

• 
  participants in the sample return their questionnaires.

• participants answer all of the questions asked on a survey.

• 
  all participants gave the same response to a questionnaire item.

• multiple items are consistent in measuring a construct or subconstruct on a questionnaire or survey.

Multiple Choice

In terms of validity, surveys should have:

• content validity.

• face validity.

• 
  construct validity.

• all of the above

  
  

What I Have Learned

Directions: As you have learned from this lesson, answer each question comprehensively.

Open Ended

Why is it important to have a good research instrument?

Open Ended

Differentiate validity and reliability. Explain how they complement each other to make a good research instrument.

Open Ended

What I Can Do

Construct your research instruments and establish validity and reliability

Open Ended

Additional Activity