Understanding Multiple Regression Analysis

• Partial Correlation measures the relationship between two variables after the effects of other independent variables have been removed.
• Semi-partial Correlation (often called part correlation) looks at the unique contribution of a variable to the total variance.

Types of Multiple Regression

Researchers choose different regression methods based on their specific research questions and whether they want to control for certain variables.

1. Standard (Forced Entry) Regression

Also known as the "ENTER" method in software like SPSS, this approach includes all specified independent variables in the model simultaneously. The researcher decides upfront which variables belong in the equation.

2. Hierarchical Multiple Regression

In hierarchical regression, the researcher enters variables into the model in "blocks" or stages. This is used to see if a new variable explains additional variance over and above what is already explained by the previous variables. For example, you might enter "age" in Block 1 and then see if "fat-free mass" adds significant predictive power in Block 2.

3. Stepwise, Forward, and Backward Regression

These methods use algorithms to determine which variables should be in the model based on their statistical significance.

• Forward Selection: The computer starts with no variables and adds them one by one, choosing the one that most improves the model until no more significant variables can be added.
• Backward Elimination: The computer starts with all variables and removes the least significant ones one by one.
• Stepwise Regression: A hybrid approach where variables are added or removed based on their contribution to the model. In health and kinesiology, researchers often use either the "ENTER" or "STEPWISE" method.

Exam Tip: When using Hierarchical Regression, always check the $R^2$ change. The increase in $R^2$ when adding a new block must be statistically significant; otherwise, the additional variable is not providing meaningful predictive value.

Assumptions and Potential Pitfalls

For a multiple regression model to be valid, several assumptions must be met:

1. Sample Size: A common rule of thumb is at least 20 participants for every one independent variable.
2. Absence of Outliers: Outliers can disproportionately influence the results and skew the regression line.
3. Normality of Residuals: The errors (residuals) should be normally distributed.
4. Homoscedasticity: The variance of the residuals should be constant across all levels of the predicted values.
5. Absence of Multicollinearity: This is a critical concern in multiple regression.

Understanding Multicollinearity

Multicollinearity occurs when two or more independent variables are highly correlated with each other. This is problematic because it becomes difficult for the model to distinguish the unique contribution of each variable. For instance, if you include both "weight" and "fat-free mass" in a model, the slope coefficient for weight might change drastically because the two variables are providing redundant information.

To detect this, researchers use a Correlation Matrix to check the relationships between all IVs. A common threshold is that correlations ($r$) should be less than 0.9 to avoid issues. Another diagnostic tool is the Variance Inflation Factor (VIF); a VIF greater than 10 is generally considered an indicator of high multicollinearity.

Working with Different Data Types

While many predictors are continuous (like age or weight), researchers often encounter categorical data. To include these in a regression, we use Dummy Coding.

A dummy variable converts a categorical variable into a series of 0s and 1s. For example, to include "sex" in a model, you might code men as 0 and women as 1. If a variable has three or more categories, you must create multiple dummy variables to represent the different groups.

Evaluating Prediction Accuracy

Once a model is built, we use it for estimation—predicting an unknown value for a specific individual.

Standard Error of the Estimate (SEE)

The SEE represents the "error of prediction" or the standard deviation of the residuals. It tells us how much the actual observed values typically deviate from the values predicted by the regression equation. A smaller SEE indicates a more accurate model.

Cross-Validation and Error Metrics

To ensure a model works on people other than those used to create it, researchers use Cross-Validation. This involves developing the equation with one half of the sample and testing it on the other half. During this process, two specific error types are calculated:

• Constant Error (CE): The difference between the mean of the actual values ($Y$) and the mean of the predicted values ($Y_P$).
• Total Error (TE): A combination of both SEE and CE, often considered the single best evaluator of regression equations.

Summary Table: Evaluating Model Accuracy