David A.
Kenny
August 9, 2004
Review of Multiple
Regression
Y = a + bX + cZ + e
Y criterion variable
X predictor variable
a intercept: the predicted value of Y when all the predictors are
zero
b regression coefficient: how much of a difference in Y results
from a one unit difference in X
e residual
Y' predicted Y given X and Z or a + bX + cZ
R multiple correlation: the correlation between Y and Y'
The coefficients (a, b, and c) are chosen so that the sum of squared errors
is minimized. The estimation technique is then called least squares or
ordinary least squares (OLS). Given the criterion of least squares, the
mean of the errors is zero and the errors correlate zero with each predictor.
If the predictor and criterion variables are all standardized, the
regression coefficients are called beta weights. A beta weight equals the
correlation when there is a single predictor. If there are two or predictors,
a beta weights can be larger than +1 or smaller than -1.
The predictors in a regression equation have no order and one cannot be
said to enter before the other.
Generally in interpreting a regression equation, it makes no scientific
sense to speak of the variance due to a given predictor. Measures of
variance depend on the order of entry in step-wise regression and on the
correlation between the predictors. Also the semi-partial correlation or
unique variance has little interpretative utility.
The standard test of a specified regression coefficient is to determine
if the multiple correlation significantly declines when the predictor
variable is removed from the equation and the other predictor variables
remain. In most computer programs this is test is given by the t or
F next to the coefficient.
Multicollinearity
If two predictors are highly correlated or if one predictor has a large
multiple correlation with the other predictors, there is said to be
multicollinearity. With perfect multicollinearity (correlations of plus
or minus one), estimation of regression coefficients is impossible.
Multicollinearity results in large standard errors for coefficients and so
a statistically significant regression coefficient is difficult (power is
low).
Another site that more extensively describes multiple regression: Statsoft
Go to the next
page.
Go to the SEM
page.