Skip to main content

Reliability and Validity in Measurement Study Guide

This guide covers the ethical considerations in psychological research, focusing on risk assessment. It details how researchers and Institutional Review Boards (IRBs) evaluate potential physical, psychological, and social risks to participants against the scientific benefits of a study. Key topics include categorizing risks, the IRB review process, and practical examples of balancing risk and benefit to ensure ethical research practices.

Save this study guide to your study set

Sign up free

Your learning path

Foundations of Psychological Research and Scientific Methods

Ethical Principles in Psychological Research

This content covers ethical principles for research with humans and animals, including autonomy, beneficence, justice, and the three R's.

  • Replication
  • p-hacking
  • Underreporting nonsignificant effects
  • Tri-Council Policy Statement (TCPS)
  • Institutional Review Board (IRB)
  • Autonomy
  • Beneficence
  • Justice
  • Informed consent
  • Debriefing
  • Vulnerable populations
  • Community-Partnered Research
  • Indigenous Peoples
  • Canadian Council on Animal Care (CCAC)
  • Replacement
  • Refinement
  • Reduction
  • Animal welfare
  • Animal rights

Limits of Intuition and Experience

Psychology systematically collects and averages evidence to overcome the limitations of intuition and experience.

  • experience
  • intuition
  • control group
  • confirming evidence
  • disconfirming evidence
  • science
  • manipulation

The Scientific Method and Theory-Data Cycle

Psychologists use the scientific method and the theory-data cycle to base beliefs on systematic, objective observation of the world.

  • Empiricism
  • Scientific method
  • Theory-data cycle
  • Theory
  • Hypothesis
  • Data
  • Replication
  • Peer-review
  • Journal

Variables and Operationalizing Research

Psychological researchers study variability by defining measured and manipulated variables through operational definitions.

  • Variable
  • Measured variable
  • Manipulated variable
  • Operational definition
  • Operationalizing
  • Self-reporting
  • Direct observation
  • Physiological measures

Descriptive Research Methods in Psychology

Psychologists use observational research methods such as naturalistic observation and case studies to measure variables of interest.

  • Observational research
  • Big Data
  • Naturalistic observation
  • Case study
  • Generalizability

Fundamentals of Experimental Research Design

Experimental research involves manipulating an independent variable to measure its effect on a dependent variable to provide evidence for causation.

  • Experimental research
  • Independent variable
  • Dependent variable
  • Temporal precedence
  • Third-variable problems
  • Random assignment
  • Experimental group
  • Control group
  • Placebo condition
  • Double-blind procedure
  • Bias

Research Foundations

Sampling and Populations

Learning the distinction between a sample and a population and how to generalize research findings.

Anecdotal Evidence vs. Science

Evaluating why personal stories and anecdotes are unreliable for scientific generalization.

Demand Characteristics

Identifying how participant expectations and cues from experimenters can bias results.

Experimental Control

Understanding the importance of minimizing confounding variables to ensure valid results.

Operational Definitions

Learning how to create precise, objective statements that describe the exact procedures used to measure abstract variables.

Peer Review Process

Understanding how experts act as gatekeepers to ensure research quality and validity before publication.

Research Quality Standards

Understanding the characteristics that make scientific research valid, reliable, and credible.

Validity and Reliability

Distinguishing between whether a study measures what it claims to (validity) and whether it produces consistent results (reliability).

Replicability in Research

Understanding why the ability for other researchers to repeat a study and find similar results is a hallmark of good science.

Objectivity in Measurement

Learning how consistency across different instruments and observers ensures research is not subjective.

Generalizability of Findings

Exploring how research results can be applied to situations, individuals, and events beyond the specific study sample.

Bias Reduction Techniques

Studying methods used to minimize influence from both participant expectations and researcher tendencies.

Scientific Principles

Objectivity vs. Subjectivity

Learning to distinguish between facts that are independent of personal belief (objective) and observations shaped by mood or experience (subjective).

Defining Variables

Understanding what a variable is—the specific object, concept, or event being measured in a study.

Measuring Behaviour

Exploring why measuring observable behaviour is more objective than attempting to measure internal thoughts or feelings.

Psychological Research Methods and Statistical Analysis

Describing Data Central Tendency and Variability

Researchers use descriptive statistics to quantify behaviour through measures of central tendency, variability, and effect size.

  • Descriptive statistics
  • Frequency distribution
  • Mean
  • Median
  • Mode
  • Variability
  • Range
  • Standard deviation
  • Effect size
  • Correlation coefficient
  • Scatterplots
  • Experimental effect size

Inferential Statistics and Statistical Significance

Inferential statistics use rules of logic and probability to estimate whether sample results represent a true relationship in a population.

  • Inferential statistics
  • Statistical significance
  • Null hypothesis
  • P value
  • Meta-analysis
  • Replication
  • Open science practices
  • Preregistration

Psychological Research Methods, Validity, and Ethics

This material covers the theory-data cycle and psychological research methods including descriptive, correlational, and experimental designs, while assessing construct, external, and internal validity and adhering to ethical principles.

  • experimental method
  • causal claims
  • random sampling
  • random assignment
  • independent variable
  • dependent variable
  • descriptive research
  • correlational research
  • theory-data cycle
  • construct validity
  • reliability
  • external validity
  • generalizability
  • internal validity
  • confound
  • empirical studies
  • Scientific method
  • Hypotheses
  • Replication
  • Variables

Descriptive Research and Survey Methods

Descriptive research uses survey research based on self-reports to measure variables within a sample from a population of interest.

  • Descriptive research
  • Correlational research
  • Experimental research
  • Survey research
  • Self-reports
  • Sample
  • Population of interest
  • Random sampling
  • Generalization

Understanding Correlational Research and Causation

Correlational research measures variables to observe relationships using scatterplots, though correlation does not imply causation.

  • correlational research
  • scatterplot
  • positive correlation
  • negative correlation
  • zero correlation
  • prediction
  • correlation is not causation
  • temporal precedence
  • third-variable problem

Research Methods

Correlational Research

Studying the relationship between variables using correlation coefficients and understanding directionality.

Correlation Coefficients

Interpreting the strength and direction (positive vs. negative) of associations.

Experimental Method

Mastering the only method capable of determining cause-and-effect relationships.

Observational and Survey Methods

Understanding naturalistic observation and survey techniques for descriptive research.

Control Groups

Mastering the role of the group that receives no treatment to provide a baseline for comparison.

Independent and Dependent Variables

Understanding how the manipulation of one variable affects the outcome of another in experimental designs.

Case Study Method

Learning how in-depth, long-term studies of specific individuals or groups provide rich qualitative data.

Random Assignment

Understanding how assigning participants to groups by chance helps ensure group equivalence.

Double-Blind Controls

Mastering the technique where neither participants nor researchers know the group assignments to prevent bias.

Placebo Effect

Understanding how participant expectations of a 'fake treatment' can influence experimental outcomes.

Laboratory vs Naturalistic Settings

Comparing the control of laboratory environments with the ecological validity of naturalistic observation.

Correlational vs Experimental

Comparing the predictive nature of correlation with the causal power of experimental design.

The Third Variable Problem

Understanding why correlation does not equal causation due to potential confounding variables.

Establishing Causality

Learning the specific conditions required in an experiment to prove that one variable causes another.

Controlled Conditions

Understanding how tightly controlled environments allow researchers to isolate specific variables.

Self-Report Data Methods

Understanding how surveys and questionnaires allow participants to provide their own observations and data.

Research Quality

Reliability and Consistency

Understanding how to determine if a measurement provides stable and consistent results across multiple observations.

Validity Concepts

Learning the degree to which an instrument or procedure actually measures what it claims to measure.

Ecological Validity

Understanding the specific challenge of applying laboratory-based findings to natural, real-world environments.

Replication and Peer Review

Studying how repeating studies and expert critique serve as the self-correcting mechanisms of science.

Ethics in Research

Risk Assessment

Evaluating the balance between potential participant risk and the scientific benefits of a study.

Use of Deception

Understanding when and why deception is considered acceptable in psychological research.

Participant Protection

Reviewing the standards for cognitive and emotional safety in human research.

Ethical Justification

Learning how researchers justify the necessity of their methods against ethical constraints.

Informed Consent

Learning the ethical requirement that participants must understand the purpose, tasks, and risks of a study before agreeing to join.

Post-Deception Ethics

Learning the requirement to debrief participants and obtain consent after using deception in a study.

Justifying Deception

Understanding the specific ethical criteria under which deception is permitted in psychological research.

Ethics Review Boards

The role of institutional oversight in approving research designs and protecting human subjects.

Participant Anonymity

The importance of maintaining privacy and confidentiality to uphold ethical standards.

Data Retention and Security

Learning the ethical and scientific necessity of keeping research data for a set period for verification and replication.

Research Designs

Descriptive Research Methods

Learning about case studies, naturalistic observation, and surveys used to describe characteristics without explaining causes.

Case Study Limitations

Analyzing why in-depth studies of single individuals may lack generalizability to the wider population.

Naturalistic Observation

Understanding the process of unobtrusively recording behavior in its natural context.

Quasi-Experimental Designs

Distinguishing between true experiments and designs where random assignment is impossible due to pre-determined characteristics.

Statistical Analysis

Measures of Central Tendency

Calculating and choosing between Mean, Median, and Mode based on data distribution.

Normal Distribution

Understanding how the mean serves as the best measure in a symmetrical distribution.

Measures of Variability

Using standard deviation to measure how 'spread out' data points are from the average.

Statistical Significance

Determining if differences between groups are likely due to the experimental variable or random chance.

Skewed Distributions

Understanding how non-symmetrical data affects the choice of central tendency measures.

Statistical Power and Sample Size

How increasing the number of participants enhances the likelihood of finding statistically significant results.

Median in Skewed Data

Why the median is often a superior measure of central tendency when data is not normally distributed.

Significance Testing Logic

The conceptual basis for determining if research findings are meaningful or due to chance.

Variability in Distributions

Understanding how dispersed scores are within a distribution and how this relates to the clustering of data.

Research Biases

The Hawthorne Effect

Recognizing when behavior changes simply because a person knows they are being observed.

Social Desirability Bias

Learning why participants may respond in ways that make them appear more favorable to researchers.

Illusory Correlations

Identifying the psychological tendency to perceive relationships between variables where none actually exist.

Ethics and Oversight

Research Ethics Boards (REB)

Understanding the role of institutional committees in weighing risks against benefits and protecting volunteers.

Risk vs. Benefit Analysis

Learning how researchers and boards justify the use of certain methods based on the potential scientific gain versus participant stress.

Anonymity and Confidentiality

Exploring how protecting participant identity helps reduce biases like social desirability.

Flawed Evidence Types

Appeals to Authority

Understanding why an expert's claim is not sufficient evidence without empirical testing and checking for agendas.

Appeals to Common Sense

Recognizing that intuitive or 'obvious' ideas must still be tested against the scientific method.

The Scientific Method

Comparing untested theories against those based on rigorous hypothesis testing.

Make it your own.

Keep these readings and topics in a new study set, or add them to one you already use.

Get started freeLog in