Introduction to Statistical Hypothesis Testing

The relationship between these two is mathematically strict: if $H_0$ contains equality ($=$, $\geq$, or $\leq$), then $H_a$ must contain inequality ($\neq$, $<$, or $>$), and vice versa.

Directional vs. Non-Directional Tests

Hypotheses are categorized based on whether the researcher is looking for a difference in a specific direction or just any difference at all.

One-Tailed Hypotheses

One-tailed tests are used when the alternative hypothesis points in a single direction—either "greater than" or "less than".

• Lower Tail: Used when the researcher suspects the parameter is less than a certain value.
  • $H_0: \mu \geq \text{value}$
  • $H_1: \mu < \text{value}$
• Upper Tail: Used when the researcher suspects the parameter is greater than a certain value.
  • $H_0: \mu \leq \text{value}$
  • $H_1: \mu > \text{value}$

Two-Tailed Hypotheses

Two-tailed tests are used when the researcher wants to test for any difference, regardless of direction. This is often used in "is/is not" or "equal/not equal" scenarios. For example, if a snack box claims to have exactly 100 calories, a two-tailed test would check if the actual mean is simply not equal to 100.

• $H_0: \mu = \text{value}$
• $H_1: \mu \neq \text{value}$

Understanding Errors in Decision Making

Because we are using samples to make guesses about entire populations, there is always a risk of making the wrong conclusion. These mistakes are categorized as Type I and Type II errors.

Type I Error

A Type I Error occurs when we reject the null hypothesis even though it is actually true. This is often linked to the significance level ($\alpha$). A classic example is a medical test telling a person they are pregnant when they are actually not.

Type II Error

A Type II Error occurs when we accept (fail to reject) the null hypothesis even though it is actually false. This is a "false negative"—failing to find a symptom or effect that actually exists. An example would be a doctor telling a visibly pregnant woman that she is not pregnant.

Exam Tip: Remember that Type I is a "False Positive" (claiming something happened when it didn't) and Type II is a "False Negative" (claiming nothing happened when it actually did).

The P-Value: Measuring Evidence

The p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true. It tells us how "surprising" our sample data is.

• Small p-value ($\leq \alpha$): The result is highly significant. We reject the null hypothesis.
• Large p-value ($> \alpha$): The result is not statistically significant. We fail to reject (accept) the null hypothesis.

Testing Two Populations

Sometimes, we aren't just testing one group; we want to compare two different populations, such as comparing the stock values of Alphabet vs. Apple or the GPA of freshmen vs. seniors.

Independent Samples

When two samples are collected separately and independently from two distinct populations, they are called Independent Simple Random Samples. To test the difference between their means ($\mu_1 - \mu_2$), we use the difference between their sample means $(\bar{X}_1 - \bar{X}_2)$ as our point estimator.

When performing a t-test for two populations where the variance ($\sigma$) is unknown, we must consider the data's spread:

• Equal Variance: The data spreads in a similar pattern across both samples.
• Unequal Variance: One sample shows a significantly wider or different spread than the other.

Matched (Paired) Samples

In other cases, the samples are not independent. A Matched Sample occurs when each sampled item provides a pair of data values. A common example is a "Pre-Test" and "Post-Test" design, such as measuring a pet's weight before and after a diet intervention. Because the data points are linked, matched samples typically result in a smaller sampling error than independent designs.

Measuring Effect Size

Even if we reject the null hypothesis, we must ask: How much does it matter? This is where Effect Size comes in. Using Cohen's $d$, we can categorize the magnitude of the difference between two means:

$$d = \frac{\bar{x}_1 - \bar{x}_2}{s_{\text{pooled}}}$$

• Small Effect: $d \approx 0.2$
• Medium Effect: $d \approx 0.5$
• Large Effect: $d \approx 0.8+$

By combining hypothesis testing with effect size, researchers can determine not only if a result is statistically significant but also if it is practically meaningful in the real world.