Understanding OLS Estimator Variance and Homoskedasticity

1. Error Variance ($\sigma^2$): A larger error variance increases the variance of the estimator. This reflects greater "noise" or uncertainty in the relationship between $y$ and $x$.
2. Variation in $x$: An increase in the total variation of the regressor $x$ (the denominator) decreases the variance of $\hat{\beta}_2$. Essentially, more variation in $x$ provides more information about the underlying relationship.
3. Sample Size ($n$): As the sample size increases, the total variation in $x$ typically increases, which subsequently reduces the variance of the estimator.

Estimating $\sigma^2$ and Standard Errors

In real-world research, the true population error variance $\sigma^2$ is unknown. We must instead use an unbiased estimator, denoted as $s^2$: $s^2 = \frac{1}{n-k} \sum_{i=1}^{n} \hat{u}_i^2$

Here, $k$ represents the number of $\beta$ parameters in the model, and $n-k$ represents the degrees of freedom. Once we have estimated $s^2$, we can calculate the standard error of our coefficients by taking the square root of the estimated variance. These standard errors are the foundation for calculating $t$-statistics and determining $p$-values to test for statistical significance.

Exam Tip: Remember that the degrees of freedom ($n-k$) decreases as you add more independent variables to your model. This is the "penalty" you pay for increasing model complexity.

Homoskedasticity vs. Heteroskedasticity

The validity of the standard OLS variance formulas depends entirely on the assumption of homoskedasticity—the idea that the "spread" of the error term is constant.

When this assumption is violated, we encounter heteroskedasticity. This occurs when the conditional variance of the error term depends on the value of an independent variable.

A classic example is the relationship between income and savings. Families with very low incomes have little discretion; they must spend most of their income on necessities, leading to low variation in savings. However, high-income families have much more discretion—some may save aggressively while others spend lavishly—leading to much higher variation in savings behavior.

If heteroskedasticity is present, the standard OLS formulas for variance become invalid. While the $\hat{\beta}$ estimates themselves may remain unbiased, the calculated standard errors will be incorrect, which can lead to misleading $t$-statistics and incorrect conclusions about whether a variable is statistically significant.

Omitted Variable Bias and Error Term Correlation

A major threat to the reliability of OLS is the violation of the Zero Conditional Mean assumption, which states that $E[u|x] = 0$. This assumption implies that the error term $u$ is not correlated with the regressor $x$ (i.e., $Cov(x, u) = 0$).

If we omit a variable from our model that is both a determinant of $y$ and correlated with $x$, we encounter omitted variable bias. In this scenario, the effect of the omitted variable is "soaked up" by the error term, creating a correlation between $x$ and $u$.

Worked Example: Education and Wages

Consider a model: $wage = \beta_1 + \beta_2 educ + e$. If we want to measure the effect of education on wages, we must ensure that the error term $e$ (which contains unobserved factors like "innate ability") is not correlated with education.

However, if individuals with higher innate ability tend to seek more education, then $e$ and $educ$ are correlated. In this case, the OLS estimate $\hat{\beta}_2$ will not just capture the effect of education; it will also capture part of the effect of innate ability, leading to a biased estimate of the true return to schooling.

Summary Table: Impact of Assumptions

\hat{\beta}$ is biased and inconsistent | | Var(u|x) = \sigma^2$| Heteroskedasticity |$\hat{\beta}$ is unbiased, but standard errors are wrong |