WorksheetsResearch Methods Final Exam Study Guide
Total questions: 99
Worksheet time: 50mins
Which of the following is an example of 'Tenacity' as a way of knowing?
A) Seeing, doing, or otherwise being exposed to a given thing repeatedly.
B) Because Taylor Swift says the earth is flat it is.
C) Taylor (non-authority) can punch a chair, then her hand will hurt.
D) If the classroom becomes silent, Tyler will ask a question because he has unmanaged anxiety.
Which way of knowing is described as 'because Taylor Swift says the earth is flat it is'?
Tenacity
Authority
Reason/Logic/Intuition
Experience/Observation
Fill in the blank: 'If the classroom becomes silent, Tyler will ask a question because he has unmanaged anxiety.' This is an example of _________.
experience/observation
prediction
hypothesis
experiment
Which of the following is NOT a strength of the scientific method?
A) We study the weaknesses of our studies
B) Less vulnerable to errors of any given mistake
C) It is based on authority
D) Allows for public verification through peer review
Which step is NOT part of the scientific method?
A) Observation
B) Hypothesis
C) Experiment
D) Memorization
Fill in the blank: The process of assigning number values to objects/characteristics is called ________.
Measurement
Evaluation
Estimation
Classification
Which level of measurement is described as 'categories' and includes examples such as gender?
Nominal
Ordinal
Interval
Ratio
What is the main goal of the falsification principle in the scientific method?
Trying to disprove alternative theories
Proving all theories to be true
Ignoring evidence that contradicts a theory
Accepting theories without question
A research hypothesis is a statement about:
The theoretical relationship between two or more variables
Assigning number values to objects
The process of public verification
Systematic observation
Which type of data provides rank order information?
A) Ordinal
B) Interval
C) Ratio
D) Nominal
Which of the following is an example of interval data?
Hurricanes category 1, 2, etc
Pain on a scale
Heart rate
Z scores
Which type of data has equal intervals between numbers and 0 is meaningful?
Ordinal
Interval
Ratio
Nominal
Fill in the blank: Transformations are when we adjust our scores (standardizing scores, turning them into percentiles) e.g., ________ and percentiles.
Z scores
Raw scores
Mean scores
Standard deviations
What is confirmation bias?
Believing something is true and only reading news articles that support this belief.
Ignoring all news articles regardless of belief.
Believing something is false and only reading news articles that support this belief.
Reading every news article without forming any belief.
Attributional bias is when you get an A on an exam and believe it’s because you’re smart, but if someone else gets an A you think it’s because they’re lucky.
True
False
What is the halo effect?
Attributing overwhelmingly good (or bad) attributes based on a single attribute, e.g., pretty people are good people.
A psychological phenomenon where people remember only negative experiences.
A marketing strategy to promote products using celebrities.
A cognitive bias that makes people forget details over time.
Which bias refers to remembering information that you have most recently read or heard?
Primacy effect
Recency effect
Extreme response
Confirmation bias
Reliability refers to the consistency and precision of the measurement instrument.
True
False
Content validity is how well an item is being measured, considering domain specification.
True
False
Criterion validity measures whether your measure (conceptualization of a given construct) effectively measures what you want it to measure.
True
False
Construct validity asks: Does what you are measuring actually measure what is being said to be measured?
True
False
Convergent validity is how correlated a given construct is to another accepted construct.
True
False
What is discriminant validity?
Discriminant validity is the extent to which a test does not correlate with things not being measured (e.g., a measure of depression should not correlate with measures of anxiety).
Discriminant validity is the extent to which a test correlates highly with similar constructs.
Discriminant validity is the ability of a test to predict future performance.
Discriminant validity is the consistency of a test over time.
Which of the following is expected if discriminant validity is present?
High correlations between distinct values
Low correlations between distinct values
No correlation between any values
High correlation within the same value
In the MTMM matrix, what does the major diagonal of the large triangle represent?
Reliability (monotrait-monomethod values)
Validity (heterotrait-monomethod values)
Correlation between different traits (heterotrait-heteromethod values)
Method variance (monotrait-heteromethod values)
In the MTMM matrix, which type of validity is represented by the minor diagonal starting from the second method level and proceeding in parallel to the major diagonal?
Convergent validity (monotrait-heteromethod values)
Discriminant validity (heterotrait-monomethod values)
Reliability (monotrait-monomethod values)
Face validity
In the MTMM matrix, what do triangles next to the minor diagonal represent?
Discriminant Validity (heterotrait-heteromethod triangles)
Convergent Validity (monotrait-heteromethod triangles)
Reliability (monotrait-monomethod triangles)
Face Validity (random triangles)
According to the APA Dictionary, what is a nomological network?
A nomological network is a broadly integrative theoretical framework that identifies the key constructs associated with a phenomenon of interest and the associations among those constructs.
A nomological network is a statistical method used to analyze the reliability of psychological tests.
A nomological network is a type of experimental design used to control for confounding variables.
A nomological network is a computer algorithm for predicting behavioral outcomes.
What does the nomothetic span represent?
Nomothetic span represents data collected (instrument validity rejected or supported by collected data over time).
Nomothetic span represents the average score of a population in a single test.
Nomothetic span refers to the ability to generalize findings to a specific individual.
Nomothetic span is the measure of internal consistency within a single test.
What is the purpose of the correction formula in restriction of range/variance?
To utilize the full study population and bring the criterion validity coefficient back up when only a subset of data is observed.
To increase the sample size artificially for statistical analysis.
To eliminate all outliers from the dataset.
To ensure the mean and standard deviation are equal in the restricted sample.
What does internal validity refer to?
The ability to generalize results to other settings
The degree to which the independent variable(s) cause the change in the dependent variable(s)
The consistency of a measure
The correlation between two unrelated variables
Which of the following is a threat to internal validity?
A) History/maturation
B) High reliability
C) Convergent validity
D) Nomothetic span
What is the threat to internal validity that occurs in a test-retest situation where there’s a break between the first and second test, and must account for any shared experiences or natural maturation between tests?
Maturation
Instrumentation
Selection Bias
Testing Effect
Which threat to internal validity must account for any familiarity that the participants may have with the test instrument, such as training participants on a test before data collection?
Testing/practice effects/instrumentation
Selection bias
Maturation
History
Select people for extreme standing in something (either really low or really high) and then that will lead to regression to the mean. What is this threat to internal validity called?
Statistical regression to the mean
Selection bias
Testing effect
Instrumentation
The sample makeup (location, age, gender, etc.) will impact internal validity. What is this threat to internal validity called?
Selection
Maturation
Instrumentation
Testing
People who stay in the study vs. drop out of the study may influence internal validity. What is this threat to internal validity called?
Mortality/attrition
Selection bias
Testing effect
Instrumentation
Talking about information specific to a condition with a participant that was not a part of that condition is called:
Diffusion
Contamination
Deception
Randomization
Change in motivation and engagement of control group relative to experimental group is called:
Compensatory rivalry
Placebo effect
Selection bias
Hawthorne effect
John Henry effect refers to:
John Henry worked on a railroad and laid track faster than the machines which ultimately killed him.
A psychological effect where people perform worse when observed.
A statistical error in experimental design.
A phenomenon where machines outperform humans in manual labor.
Sometimes participants effects (reactivity) or experimenter gives something away that cues the participant on how to respond. What is this called?
Reactivity/demand characteristics
Placebo effect
Confirmation bias
Double-blind procedure
Which type of subject wants to help validate the researchers’ hypothesis and behaves accordingly, even though they have not been told what it is?
The good subject
The naive subject
The skeptical subject
The disinterested subject
Which type of subject believes they have to act uninformed and be molded/changed by the study?
The faithful subject
The skeptical subject
The disinterested subject
The resistant subject
Which type of subject is upset that they have to participate in the study so they do what they can to mess it up?
Negativistic subject
Compliant subject
Indifferent subject
Curious subject
Which type of subject is very nervous and careful about what they say/do?
Apprehensive subject
Confident subject
Carefree subject
Indifferent subject
What is the Rosenthal effect in experimental research?
When the experimenter projects expectancies onto participants based on internal beliefs
When participants act as their own control group
When two or more extraneous variables overlap
When a research feature influences the dependent variable
Fill in the blank: ________ occurs when two or more extraneous variables (EVs) or possible outcomes overlap so much that the effects cannot be differentiated from one another.
Confounds
Controls
Variables
Biases
Which of the following best describes an artifact in research?
A) An alternative explanation for the effect
B) A research feature (other than the IV) that influences the DV
C) A separation between theory creator and experimenter
D) A type of experimental design
What is the difference between confounds and artifacts?
Confounds are alternative explanations for the effect, whereas artifacts are specific to the research design that leads to a factor impacting the DV other than the manipulated variable (IV).
Artifacts are alternative explanations for the effect, whereas confounds are specific to the research design.
Both are the same.
Confounds only occur in true experimental designs.
Fill in the blank: In a ________ design, participants act as their own control/comparison group (i.e., pre- and post-test).
within-subjects
between-subjects
cross-sectional
longitudinal
Which type of design uses aspects of both between- and within-subjects designs and uses a mixed ANOVA?
True experimental design
Mixed design
Between-subjects design
Within-subjects design
In true experimental designs, one variable must be manipulated (IV) and subjects are randomly assigned to conditions.
True
False
Fill in the blank: In a lab setting, the researcher can control ________ which increases internal validity.
EVs (extraneous variables)
DVs (dependent variables)
IVs (independent variables)
CVs (controlled variables)
What is the purpose of including manipulation checks in an experiment?
To identify if the independent variable (IV) is doing what you expected it would.
To increase the sample size of the experiment.
To ensure the results are statistically significant.
To measure the dependent variable more accurately.
In a treatment control, post only design (t-test, DV mean between 2 groups), what is the main advantage?
Can make a causal claim because any differences can be directly linked to IV.
It allows for the measurement of pre-existing differences between groups.
It eliminates the need for random assignment.
It provides information about long-term effects of the treatment.
Which experimental design allows you to compare all means and the grand mean using one way ANOVA?
Multiple group treatment – control, post only.
Single group pre-post design.
Matched pairs design.
Repeated measures design.
In a treatment – control, pre-post design, what is the purpose of the pre-measurement of the dependent variable (DV)?
To see if that phenomenon was already happening/assessing how sensitizing a test is.
To eliminate the need for a control group.
To ensure random assignment of participants.
To measure the effect of the independent variable before the intervention.
What does ANCOVA help to adjust for in experimental designs?
Covariates (confounds).
Sample size.
Randomization.
Measurement error.
Which design examines the effects of multiple independent variables and their interactions?
Factorial designs (factorial ANOVA).
Repeated measures design.
Single-subject design.
Correlational design.
Which experimental design is a combination of #1 and #3 and is not used frequently?
Soloman 4 group.
Randomized block design.
Latin square design.
Cross-over design.
What is measured in aptitude-treatment interactions?
A variable that is believed to be relevant to the IV/DV connection.
The average score of all participants.
The time taken to complete the experiment.
The number of treatments administered.
Which test compares the means of two groups or scores and can use independent or paired samples?
T-test.
Chi-square test.
ANOVA.
Regression analysis.
Which analysis plan compares the means of three or more groups to examine an IV's main effect?
One way ANOVA.
Paired t-test.
Chi-square test.
Correlation analysis.
Which analysis plan examines the main effects and interaction of two or more IVs?
Factorial ANOVA.
Simple Linear Regression.
One-way ANOVA.
Chi-square Test.
What is a limitation mentioned for participants in experimental designs?
Might not have access to the target population.
May always receive the treatment.
Are guaranteed to be unbiased.
Can only participate in surveys.
What is a key ethical limitation of quasi-experimental designs?
Cannot operationalize or randomize all possible variables
Cannot measure outcomes
Cannot use control groups
Cannot analyze data
In quasi-experimental designs, what does ecological validity refer to?
The ability to generalize findings to real-life situations outside of a lab setting
The ability to randomize participants
The ability to use multiple baselines
The ability to collect qualitative data
Fill in the blank: In quasi-experimental designs, there is at least one IV; ______ assignment to groups is not used.
random
systematic
voluntary
alphabetical
If differences are found between groups in a quasi-experimental design, what statistical method should be used to control for known differences?
A) ANOVA
B) ANCOVA
C) T-test
D) Regression
Which of the following best describes an ABAB design in interrupted time series?
One group is measured repeatedly, pre and post IV manipulation
Multiple baseline and treatment assessments
Only post-test measurements are taken
No control group is used
In single case research designs, what is a primary characteristic of observational studies?
Collect a lot of data from one person
Use random assignment
Only use qualitative data
Do not implement interventions
Fill in the blank: In single case research designs, qualitative data analysis involves identifying patterns that match with ______ or are inferred from the data.
theoretical predications/hypotheses
statistical averages
random variables
experimental errors
Which of the following is a pro of using nonexperimental designs?
Low reliability
Data is basically unlimited
Data interpretations can be unclear
Not as useful for hypothesis/theory testing
Which of the following is NOT a con of nonexperimental designs?
Low reliability, generalizability to broader population, and validity
Data interpretations can be unclear
Useful for descriptions, hypothesis generation, examining rare phenomena
Not as useful for hypothesis/theory testing
Nonexperimental designs cannot declare ______, only correlation.
causation
association
variation
prediction
Which of the following is NOT a type of nonexperimental design purpose?
Descriptive
Comparative
Experimental
Predictive
Which method is described as using publicly available data such as census or newspaper records?
Surveys
Observational
Archival
Experimental
Quantitative analytic methods are usually based on a ______ sample and are statistically analyzed for hypothesis testing.
larger (more representative)
smaller (less representative)
random (uncontrolled)
biased (non-representative)
Which approach involves observing a group over an extended period of time?
Cross-sectional
Longitudinal
Cohort-sequential
Experimental
In ANOVA, between groups design refers to systematic variance attributable to the ______.
IVs (independent variables)
DVs (dependent variables)
random error
sample size
Significance testing in ANOVA partitions variance into signal (between groups) and ______ (within groups).
noise
error
mean
sum
What does regression examine in statistics?
A) Relationships between predictors and criterion (outcome)
B) Relationships between mediators and moderators
C) Relationships between variables and constants
D) Relationships between samples and populations
Fill in the blank: The equation for regression is specified as ___ .
Y = a (y-int) + b(slope)X + e (error terms residuals)
Y = aX + b + e
Y = bX + a
Y = a + bX
Which of the following best describes a moderator in statistical analysis?
It changes the effect of another independent variable (IV) on the dependent variable (DV) based on status
It operates through another predictor
It is always a continuous variable
It is used only in probability sampling
Fill in the blank: Full mediation occurs when one predictor is operating through another predictor, such as IQ -> Education -> Job performance.
Full mediation
Partial mediation
Direct effect
Moderation
Linearity in correlation and regression always assumes a linear correlation represented by a line.
True
False
Which sampling procedure involves selecting every person relevant to the sample population from a list and picking every nth one?
Systematic selection
Convenience sampling
Quota sampling
Snowball sampling
Fill in the blank: Convenience sampling involves a subject of a certain population agreeing to participate in the study.
Convenience sampling
Random sampling
Systematic sampling
Stratified sampling
Which sampling method is described as randomly selecting people from a list so that everyone in the study population has an equal chance of being included?
Simple random sampling
Quota sampling
Snowball sampling
Systematic selection
What is stratified random sampling?
Random sampling among multiple groups
Study population is grouped in meaningful ways
Using a random number generator
Sampling error
What is cluster sampling?
Random sampling among multiple groups
Study population is grouped in meaningful ways (where they work/live/go to school, etc)
Using a random number generator
Sampling error
Type I error is saying there is an effect when there is not an effect. What is the error rate called?
Alpha
Beta
Power
Effect size
Type II error is saying there is not an effect when there is an effect. What is the error rate called?
Alpha
Beta
Power
Effect size
Fill in the blank: Statistical power is the probability of concluding there was an effect when there was one. Power = _____
1-beta
alpha
beta
1-alpha
As alpha increases, sample size _________?
decreases
increases
remains the same
becomes zero
As power increases, sample size _________
increases
decreases
remains the same
becomes unpredictable
Which of the following is NOT a way to determine expected effect sizes?
A) Average distance reported in previous research
B) Smallest meaningful difference
C) Using Cohen’s benchmarks
D) Power = 1-beta
What does NHST debate focus on?
Identifying type 2 error rate controlled by beta
Using random sampling
Determining effect sizes
Central limit theorem
What is the most costly error in psychology according to the notes?
False negatives
False positives
Type 2 errors
Sampling errors
