Understanding Multiple Regression Analysis

Exam Tip: When asked to compare the importance of two different predictors in a model, always look at the standardized beta weights, not the unstandardized $b$ coefficients, because beta weights account for different scales of measurement.

Approaches to Building a Model

Depending on your research question, you will choose one of three main methods to enter your variables into the regression model:

1. Forced (or Standard) Regression: Often called the "ENTER" method in SPSS, all independent variables are entered into the model simultaneously.
2. Hierarchical Regression: This is used when you want to see if a new variable adds significant predictive power over and above variables already in the model. You enter variables in "blocks" and check if the change in $R^2$ is statistically significant.
3. Stepwise Regression: The computer automatically selects the variables for the model based on which ones have the strongest relationship with the dependent variable, adding or removing them one by one.

Understanding Correlations: Zero-Order, Partial, and Semi-Partial

In multiple regression, we are often interested in the "unique" contribution of a single variable after accounting for the others. To understand this, we distinguish between three types of correlations:

• Zero-order correlation: This is a standard, simple correlation between two variables without considering any others.
• Partial correlation: This measures the relationship between two variables after the effects of all other independent variables in the model have been removed.
• Semi-partial correlation: This measures the unique contribution of one variable to the total variance in the dependent variable.

Assumptions and Potential Pitfalls

For a multiple regression model to be valid, several statistical assumptions must be met:

1. Sample Size: A common rule of thumb is to have at least 20 participants for every one independent variable.
2. Absence of Outliers: Extreme values can disproportionately influence the results and skew the model.
3. Normality of Residuals: The errors (the differences between actual and predicted values) should follow a normal distribution.
4. Homoscedasticity of Residuals: The variance of the residuals should be constant across all levels of the predicted values.
5. Absence of Multicollinearity and Singularity: This is critical. Multicollinearity occurs when independent variables are too highly correlated with each other (e.g., $r > .90$), which makes it difficult to determine their individual effects. Singularity is an extreme form of multicollinearity where variables are so highly related that the math behind the regression breaks down.

Note on Multicollinearity: If you add a variable that is highly correlated with an existing one (like adding Fat-Free Mass to a model that already has Weight), you may see the slope coefficients change drastically. You can check for this using the Variance Inflation Factor (VIF); a VIF > 10 is generally considered high.

Estimation, Error, and Validation

Once a model is built, we use it for estimation—predicting unknown values. However, every prediction has error.

Measuring Error

The Standard Error of the Estimate (SEE) represents the standard deviation of the residuals (the differences between actual $Y$ and predicted $\hat{Y}$). A smaller SEE indicates a more accurate model.

When providing a prediction, it is best practice to provide a confidence interval (CI) around that estimate. For example, a 95% confidence interval is calculated as: $\text{Estimate} \pm 1.96(\text{SEE})$ For a 99% confidence interval, use 2.58 instead of 1.96.

Cross Validation

To ensure a model works on people other than those in the original study, researchers use cross validation. This involves developing the equation with one half of the sample and testing it on the other half. During this process, we calculate:

• Constant Error (CE): The difference between the mean of the actual values and the mean of the predicted values.
• Total Error (TE): A combination of both SEE and CE, often considered the single best way to evaluate a regression equation.