David A. Kenny
August 9, 2004


Review of Multiple Regression
                        Y = a + bX + cZ + e

    Y    criterion variable
    X    predictor variable
    a    intercept: the predicted value of Y when all the predictors are 
	 zero
    b    regression coefficient: how much of a difference in Y results 
         from a one unit difference in X 
    e    residual
    Y'   predicted Y given X and Z or a + bX + cZ
    R    multiple correlation: the correlation between Y and Y'

The coefficients (a, b, and c) are chosen so that the sum of squared errors
is minimized.  The estimation technique is then called least squares or 
ordinary least squares (OLS).  Given the criterion of least squares, the 
mean of the errors is zero and the errors correlate zero with each predictor.

If the predictor and criterion variables are all standardized, the 
regression coefficients are called beta weights.  A beta weight equals the
correlation when there is a single predictor.  If there are two or predictors,
a beta weights can be larger than +1 or smaller than -1.

The predictors in a regression equation have no order and one cannot be 
said to enter before the other.

Generally in interpreting a regression equation, it makes no scientific 
sense to speak of the variance due to a given predictor.  Measures of 
variance depend on the order of entry in step-wise regression and on the 
correlation between the predictors.  Also the semi-partial correlation or 
unique variance has little interpretative utility.

The standard test of a specified regression coefficient is to determine 
if the multiple correlation significantly declines when the predictor 
variable is removed from the equation and the other predictor variables 
remain.  In most computer programs this is test is given by the t or 
F next to the coefficient.

Multicollinearity
If two predictors are highly correlated or if one predictor has a large
multiple correlation with the other predictors, there is said to be
multicollinearity.  With perfect multicollinearity (correlations of plus 
or minus one), estimation of regression coefficients is impossible.
Multicollinearity results in large standard errors for coefficients and so 
a statistically significant regression coefficient is difficult (power is
low).

Another site that more extensively describes multiple regression: Statsoft

Go to the next page.

Go to the SEM page.