Revision Class 5: Research Methods

Research Methods

Overview

• Population and sample

• Variables & operational definitions

• Participant selection and random allocation

• Experimental reliability & validity

• Mean & standard deviation

• Controlling extraneous variables

• Conclusions & generalisations

Population and sample

• Population - the group of people of interest to an experimenter. It is hoped the results of the experiment can be generalised to this research population.

• Sample - a subset of the research population. commonly referred to as participants; can be biased (not representative of the research population) Link to participant selection.

• 1 quick question

Multiple Choice

Dr Leyshan was investigating the effect of alcohol on the driving ability of Victorians. She recruits 120 participants who have a Victorian drivers license by putting an advertisement on social media.

The population and sample for this experiment are, respectively

• People with a license; drivers who responded to the advertisement,

• Victorians with a driver's license; 120 drivers from Victoria.

• Victorians who responded to the advertisement; 120 Victorian drivers.

• 120 Victorian drivers; Victorians.

Variables

• IV - the variable that is manipulated or changed. It is given to the participants. There may be more than one IV in an experimentco

• DV - measures the change caused by the IV. It is usually the result of some sort of test.

• Extraneous - any variable other than the independent variable (IV) that can cause a change in the dependent variable (DV).

• Confounding - is when an EV is not controlled for, it will have systematic ‘confusing’ effects on the experiment; look for things within the design of the experiment.

• 4 quick questions

Multiple Choice

What is the DEPENDENT variable in this experiment?

Research question - Does eating breakfast improve test scores?

• eating breakfast or not

• the score received on the test

• what time you get up in the morning

• how tasty the breakfast is

Multiple Choice

A researcher is conducting a study which requires participants to name the colour of written words as quickly as possible. In one condition, the colour of the words matches the word that is written. For example, 'red' is written in red. In the second condition, the colour does not match the word written . For example, 'red' is written in blue.

The researcher records the number of errors made, and the time taken to say the colour of the word. The independent variable in this study is

• the actual colour of the word

• the number of errors made

• the time taken to say the colour of the word

• whether or not the colour matches the word

Multiple Choice

In an experiment, a variable that can cause a change in the dependent variable and therefore could affect the results of the experiment in an unwanted way is called a/an _________ variable.

• independent

• participant

• dependent

• extraneous

Multiple Choice

In an experiment, some participants took a standard printed exam; others took the exam using a computer administration procedure. Scores obtained with the two exam procedures were compared. The independent variable was __________ and the dependent variable was __________.

• exam type; exam score.

• exam score; exam type.

• standard exam; online exam

• exam type; exam grade

Operational definitions

• is the statement of procedures the researcher is going to use in order to measure a specific variable.

• allows the researchers to describe in a specific way what they mean when they use a certain term.

• define variables in a way that allows other people to see if the research has validity.

Operational definition - Example

• A researcher wants to measure if age is related to addiction. Perhaps their research question is: Does the incidence of addiction will increase with age.

• Here we have two variables, age and addiction. In order to make the research as clear as possible, the researcher must define how they will measure these variables.

• Essentially, how do we measure someone’s age and how to we measure addiction?

Example cont.,

• Variable One: Age might seem straightforward. You might be wondering why we need to define age if we all know what age is. However, one researcher might decide to measure age in months in order to get someone’s precise age, while another might choose to measure age in years. In order to understand the results of the study, we will need to know how the researcher operationalised age.

• For the sake of this example lets say: Age defined as how old someone is in years.

Example cont.,

• Variable Two: The variable of addiction is slightly more complicated than age. In order to operationalise it the researcher has to decide exactly how they want to measure addiction. They might narrow down their definition and say that addiction is defined as going through withdrawal when the person stops using a substance. Or the researchers might decide that the definition of addiction is: if someone currently meets the DSM-5 diagnostic criteria for any substance use disorder.

• Addiction as measured by the participant meeting the DSM-5 diagnostic criteria for alcohol substance use disorder.

• 3 quick questions

Multiple Choice

T or F:

Operational definitions are useful because they offer specificity, but are not essential when the scientific method is used.

• True

• False

Multiple Choice

Which of the following would not be an operational definition for the dependent variable of 'smoking less'

Smoking less as measured by

• the person not smoking a cigarette for 1 month.

• the person who has not smoked in a year.

• the cost of cigarettes.

• a 50% reduction in the number of cigarettes smoked.

Multiple Choice

An experiment was conducted to see if caffeine increases memory ability. Prior to the experiment, each participant sat a memory test (Memory Test A). After the experiment, the participants’ were administered a second memory test (Memory Test B).

The operationalised definition for the dependent variable of memory ability is - memory ability as measured by

• the score on memory test B.

• the score on memory test A.

• the difference is score between memory test A and B.

• the mean score for each memory test.

Participant selection - good, better, best

• Convenience sample - sample is selected in the quickest and easiest way by selecting the most readily available people; likely sample is not representative of the research population

• Random sample - sample is selected in a way that every person in the research population has an equal chance of being selected for the sample; if small in size may not be representative of the research population.

• Stratified sample - research population is divided into relevant groups ( strata), then the research selects a sample from each strata in the same proportion that they occur in the research population; difficult to define strata and time-consuming/expensive.

Random allocation (or assignment)

• Random- every once has an equal chance.

• Allocation (or assignment) - process of putting the participants into the groups of the experiment.

• Random allocation therefore means that everyone in the sample has an equal change of being put in the E or C group (or any of the conditions/groups in the experiment).

Why random?

• Random sampling - strength is that, providing the sample is large enough, it is more likely to be representative of the population than a convenience sample. Links to external validity and ability to generalise to the wider population.

• Random allocation - reduces the chance of the conditions being biased with some participant variable being over or under represented.

• 3 quick questions

Multiple Choice

Random sampling involves

• anyone available being selected to participate in the research.

• every member of the sample having an equal chance of being in any of the research groups.

• every member of the population having an equal chance of being in the sample.

• every member of the population having an equal chance of being in any of the research groups.

Multiple Choice

What is one limitation of using convenience sampling?

• It can be time-consuming to conduct.

• It is costly to conduct correctly.

• It is difficult to define the relevant groups and to select the participants proportionally.

• It is likely to generate more bias in the sample and therefore less likely to be representative of the population.

Multiple Choice

A research population consisted of 40 patients with mild Alzheimer's disease, 30 patients with moderate Alzheimer's disease and 30 patients with severe Alzheimer's disease.

A research requires a sample of 20 participants from this population. Using stratified sampling, how many participants from the three different degrees of Alzheimer's disease should be taken from each strata?

• 16 mild

  12 moderate

  12 severe

• 6 mild

  8 moderate

  6 severe

• 4 mild

  3 moderate

  3 severe

• 8 mild

  6 moderate

  6 severe

Experimental reliability

• refers to the likelihood that another experimenter will perform exactly the same experiment under the same conditions and generate the same, or very similar, results (within a very narrow range of values).

Validity - A measurement is 'valid'

• if it measures what it claims to be measuring; for example, a test of memory should measure memory and not something else (such as intelligence or emotional state).

• Both experimental design and the implementation should be considered when evaluating validity.

Internal validity

• refers to whether there is a causal relationship between the independent and dependent variables or whether any observed effects of the investigation or procedure are due to some other factor.

• can be improved through a number of methods including using a particular type of research design, controlling extraneous variables, using standardised instructions and procedures, counterbalancing and eliminating experimenter effects.

External validity

• refers to the extent to which the results of an investigation can be generalised to other settings, other people and over time.

• can be improved through a number of methods including conducting experiments in settings natural to the research question of interest and using random sampling to select participants.

• diagram to help understand reliability and validity then 3 quick questions.

Reliability and validity - are the measurements on target!

Multiple Choice

This is an example of...

• Reliability but not validity

• Validity but not reliability

• Neither validity or reliability

• validity and reliability

Multiple Choice

The ability of the instrument to measure what it intends to measure.

• Validity

• Reliability

Multiple Choice

The term that  refers to the consistency of results.

• Validity

• Reliability

Statistical analysis of data

• VCE Psychology students are expected to calculate mean as a measure of central tendency for a set of data.

• There is a qualitative understanding that standard deviation is used to summarise the spread of data values around the mean.

• Students should recognise that standard deviation can be useful for comparing the means and the spread between two or more population samples, particularly that in the data a higher standard deviation represents greater variation, or spread, in the data set.

• 2 quick questions

Multiple Choice

To find the _________ you add up all the numbers and then divide by how many numbers you have.

• Mean
• Median
• Mode
• Range

Multiple Choice

When scores are "bunched up"in the center of a distribution, the

• mean will be relatively low.

• standard deviation will be relatively low.

• correlation coefficient will be a negative number.

• standard deviation will be relatively high.

Controlling extraneous variables so they don't become confounding variables!

• You need to be able to identify and explain a range of variables, other than the IV, that could effect the DV/results.

• You also need to be able to describe ways to eliminate or control these extraneous variables.

Potential extraneous variables due to the participants being used.

Potential extraneous variables due to the participants cognitive processing.

Potential extraneous variable due to the experimenters themselves.

Source of extraneous variable is due to difference between the instructions given and procedures used.

Potential extraneous variable due to the type of research design used.

Multiple Choice

In a study of pain medications, injections of a saline solution (which contains no chemicals) were shown to be quite effective for relieving pain. This result is described as occurring due to

• the experimenter effect.

• the placebo effect.

• an order effect.

• a placebo.

Multiple Choice

A confounding variable created when performance of one task by participants in an experiment affects their performance in a second experimental task, is commonly referred to as a/an

• order effect.

• placebo effect.

• experimenter effect.

• participant effect.

Multiple Choice

Psychology researchers attempt to minimise or eliminate the impact of uncontrolled variables in experiments because the presence of these variables:

• Can affect the measurement of the independent variable

• Means that only the uncontrolled variables are measured

• Do not enable the researcher to control the dependent variable

• Negatively impact the generalisability of a study

Conclusions

• are made at the end of an investigation.

• should be based solely on the results of the research.

• should restate the hypothesis and say if the hypothesis is supported or not (never say the hypothesis has been proved).

What should a conclusion contain?

What thoughts the research had going into it - this means you have to restate the hypothesis or expected outcomes depending on whether the research has a hypothesis or not.

What were the results - select specific data from the results that summarise the findings. If you aren’t quoting any numbers or observations from the experiment, then you are doing it wrong.

Concluding statement - this is the part where you state whether the results supported the hypothesis or not. If your experiment was not investigating a hypothesis, but instead an objective (like how does amount of flying affect mood), then write a statement based on what you have.

Description + Conclusion/s

• This graph is showing the recommend hours of sleep that is needed for various age groups. Infants require 15 hours, 2-5 year olds 13 hours, 6-11 year olds 11 hours, teens 9 hours and adults 8 hours.

• It can be concluded from these results that it appears as a person ages the hours of sleep they need per day decreases.

Description of graph.

This graph shows that if people have less than 5 hours of exercise per week their perceived stress level is of 8 out of 10 whereas people having more than 5 hours of exercise per week perceived stress level drops by half to 4 out of 10.

Conclusion/s

It can be concluded based on these results that it appears that people who do more than five hours of exercise per week perceive their stress levels as half that of people who exercise for fewer that five hours per week.

Open Ended

What conclusion/s can the researcher make?

Generalisation

• is a decision or judgment, based on the results, about how widely the results/findings of a study can be applied to the population from which the sample was drawn.

• can not be made if the sample is not representative of the research population due; more representative the higher the ability to generalise.

• can not be made if there are confounding variables, no internal validity or when the results are not reliable.

• 2 quick questions

Multiple Choice

The results of a study are extended from a sample to the population from which this sample was taken. This process is called

• discrimination.

• generalisation.

• random assignment.

• stratified sampling.

Multiple Choice

The term 'external validity' is concerned with:

• the question of whether the results of a study can be generalised beyond the specific research context.

• whether the research question is judged to be a good research question by those outside the study.

• the question of whether or not social scientific findings are applicable to people's every day, natural social settings.

• an ambiguous concept whose meaning depends on how it is defined.

More on this answer.

This issue is concerned with the question of whether the results of a study can be generalised beyond the specific research context.

It is in this context that the issue of how people are selected to participate in research becomes crucial. External validity is one of the main reasons why researchers are so keen to generate representative samples.