Understanding Descriptive and Inferential Statistics

The Path to Inference: From Samples to Populations

When we collect data, we are usually looking at a sample—a small subset of a larger population. Because of sampling variation, different random samples from the same population will produce slightly different results. To manage this uncertainty, we use standardized scores and distributions.

Standardized Scores and Distributions

• z-score: A standardized score that tells us exactly how many standard deviations a raw score is from the mean. It is calculated as: $z = \frac{x - \text{mean}}{SD}$
• Sampling Distribution: A theoretical distribution of means created by taking thousands of random samples from a population.
• t-distribution: Used when sample sizes are small; it is a "compressed" version of the normal distribution that accounts for the extra uncertainty of small samples.

Estimating the Truth: Confidence Intervals and Standard Error

We rarely know the true population mean, so we estimate it using the Standard Error of the Mean (SEM). The SEM tells us how much the sample mean is expected to vary from the actual population mean. It is calculated as: $\text{SEM} = \frac{SD}{\sqrt{N}}$

We use this to build 95% Confidence Intervals (CI), which provide a range of values likely to contain the true population mean.

• Lower CI: $\text{Mean} - (t\text{-score} \times \text{SE})$
• Upper CI: $\text{Mean} + (t\text{-score} \times \text{SE})$

Exam Tip: If you are comparing two groups and their 95% Confidence Intervals do not overlap, the difference between them is statistically significant ($p < 0.05$). If they do overlap, the difference is likely due to chance.

Null Hypothesis Significance Testing (NHST)

NHST is the formal framework used to decide if an observed effect is "real" or just a result of sampling error.

The Logic of the Test

1. Null Hypothesis ($H_0$): The default position that there is no difference between conditions; any observed difference is due to chance.
2. Alternative Hypothesis: The position that a difference does exist.
3. Alpha Level ($\alpha$): The threshold set before the experiment (usually 0.05) to decide when to reject the null hypothesis.
4. p-value: The probability that the observed difference occurred by chance.
   • If $p < 0.05$: The result is statistically significant; we reject the null hypothesis.
   • If $p > 0.05$: The result is non-significant; we fail to reject the null hypothesis.

Dealing with Data Complications: Bias and Outliers

Real-world data is rarely perfect. Two major issues that can ruin an analysis are bias and outliers.

Bias occurs when your sample statistics are systematically different from the true population values. Outliers are scores that fall far outside the normal range of the rest of the data.

Outliers are particularly dangerous because they have a disproportionate impact on the Sum of Squared Errors (SSE). Because deviations are squared, a single extreme outlier can massively inflate the variance, standard deviation, and standard error, which then "cascades" through your entire analysis, leading to incorrect p-values and confidence intervals.

How to Detect and Fix Problems

To detect outliers, researchers use visual plots (like Boxplots or Histograms) or check if z-scores exceed certain thresholds (e.g., $|z| > 1.96$ or $|z| > 3.29$).

If your data is messy, you can use the TWAT (or WATT) methods to reduce bias:

1. Trimming: Deleting a percentage of extreme scores from the tails.
2. Winsorizing: Replacing outliers with the next most extreme non-outlier value.
3. Robust Methods (e.g., Bootstrapping): Treating the sample as the population by taking thousands of sub-samples to create a stable estimate.
4. Data Transformation: Applying mathematical functions to "squash" the data into a more normal shape.

Final Pro-Tip: While transformations are useful, modern statisticians like Andy Field recommend using robust procedures like bootstrapping whenever possible. Transformations change the very nature of what you are measuring (e.g., comparing geometric means instead of arithmetic means), which can make your results harder to interpret.