Confidence Intervals and Simple Linear Regression

The Role of Standard Error and the t-Distribution

How wide or narrow our interval is depends on the standard error. The standard error measures the variability of the sample mean and is calculated using the standard deviation ($\sigma$) and the sample size ($n$):

$\text{Standard Error} = \frac{\sigma}{\sqrt{n}}$

In many real-world scenarios, the population standard deviation ($\sigma$) is unknown. In these cases, we must use the sample standard deviation ($s$) as an estimate.

When $\sigma$ is unknown, the standard Normal distribution (the $z$-distribution) is no longer the best model because our estimate of the standard deviation might be slightly off. Instead, we use the Student’s t-distribution. The $t$-distribution takes the sample size into account; as the sample size increases, the $t$-distribution becomes more similar to the Normal distribution.

To calculate a 95% confidence interval when $\sigma$ is unknown, we use the following formula:

$\bar{y} \pm t_{crit} \times \frac{s}{\sqrt{n}}$

where $t_{crit}$ is the critical value from the $t$-distribution.

Note on Normality: You can only use the $t$-distribution if the variable of interest is approximately Normal (unimodal and symmetric). However, if your sample size is large, the Central Limit Theorem (CLT) suggests that the sampling distribution of the mean will be approximately Normal regardless of the population shape, making the $t$-distribution quite robust.

Simple Linear Regression

While confidence intervals help us estimate a single value, simple linear regression allows us to model the relationship between two different numerical variables.

Variables and Scatterplots

In regression, we deal with two types of variables:

1. Independent Variable ($X$): Also called the predictor, determinant, or response variable. This is the variable we use to make predictions.
2. Dependent Variable ($Y$): Also called the outcome or response variable. This is the variable we are trying to predict.

To visualize the relationship between these variables, we use a scatterplot. By plotting $X$ on the horizontal axis and $Y$ on the vertical axis, we can see patterns, such as whether the trend is positive (both increase together) or negative (one increases while the other decreases).

Warning about Causation: Just because a relationship exists between $X$ and $Y$ does not mean $X$ causes $Y$. They might both be influenced by a third factor, known as a confounder.

The Least Squares Regression Line

The goal of linear regression is to find the "best" straight line through the scatterplot. This is known as the least squares regression line. The equation for this line is:

$\hat{y} = b_0 + b_1x$

In this equation:

• $\hat{y}$ (pronounced "y-hat") represents the predicted value.
• $b_0$ is the $y$-intercept.
• $b_1$ is the slope.

The "best" line is mathematically defined as the one that minimizes the sum of the squared residuals. A residual ($e$) is the difference between the observed value ($y$) and the predicted value ($\hat{y}$):

$e = y - \hat{y}$

Interpreting the Slope

The most important part of the regression line is the slope, $b_1$. The slope tells us how much we expect the dependent variable to change for every one-unit increase in the independent variable:

$\text{For every 1 unit increase in } X, \text{ we expect } Y \text{ to change by } b_1 \text{ units, on average.}$

For example, if we are studying the price of used Toyota Corollas and find a slope of $-1310$, we would interpret this as: "For every extra year in the age of a used Toyota Corolla, we expect the price to decrease by an average of $1,310".