Lecture 12

(Chapter 12 continued)

Regression and Correlation Techniques:

The population regression equation is written as:

Y_i = A + B.X_i + e_i

where e_i is the error term.

The sample regression equation is written as:

Y_i^hat = a + b.X_i.

The coefficients estimators a and b are obtained by method of least squares:

b = { nSX_iY_i – (SX_i)( (SY_i)}/{nSX_i² – (SX_i)²}

a = (SY_i)/n – b. (SX_i)/n

Given data on Y and X, we know how to compute a and b.

Characteristics of Least Squares Estimates:

Optimally, the analyst would like to know the population regression line. However, this regression line can not be calculated because the statistician has only the sample observations to work with. Therefore, the best the statistician can do is to calculate the sample regression line and use it as an estimate of the population regression line.

The statistics – a and b – are estimators of A and B, the constants in the population regression. These least squares estimators have the following desirable properties:

1) Unbiasedness - the sampling distributions of a and b have means A and B respectively.

2) Efficiency – Least Squares (LS) estimators have the minimum standard deviation.

3) Consistency – As the sample size becomes larger and larger, the values of a homes in on A and values of b homes in on B.

Because of these properties, LS estimator is widely used.

The Standard Error of Estimate

Recall from Diagram 12.3 that the standard deviation of the conditional probability distribution of Y_i is assumed to be the same independent of X_i . This standard deviation, denoted by s_e, is a measure of the amount of scatter about the regression line in the population. If s_e is large there is much scatter.

The sample statistic used to estimate s_e is the standard error of estimate. It is given by

s_e = { S(Y_i – Y_i^hat)²/n-2}^1/2

The above formula can be simplified as:

s_e = { (SY_i² – aSY_i - b SX_iY_i )/n-2}^1/2

and easy to calculate.

Do exercises 12.8 (a-b), 12.9 (a-b), 12.10 (a), 12.11 (a), 12.12 (a), and 12.13 (a – e).

The coefficient of determination

Once the regression line has been fitted, one would like to know how well the line fits the data.

Look at Figure 12.9.

One can logically look at the standard error of estimate s_e. But this measure should be looked at in relation to the total sum of squares in Y_i which is defined as

S (Y_i - Y^bar)²

The total sum of squares can be decomposed into two parts:

(Y_i - Y ^{ba r}) = (Y_i – Y_i^hat) + (Y_i^hat - Y ^{ba r})

Þ S (Y_i - Y ^{ba r})² = S (Y_i – Y_i^hat)² + S (Y_i^hat - Y ^{ba r})²

Þ Total variation in the dependent variable = variation in the dependent variable not explained by the regression + total variation in the dependent variable that is explained by the regression.

Coefficient of determination is defined as:

r² = (variation explained by regression/total variation in Y)

= (total variation in Y – variation not explained by regression)/ total variation in Y)

r² = 1 – (variation not explained by regression/total variation in Y)

It is the proportion of the variation in Y that is explained by the regression.

r² = [nSX_iY_i -(SY_i).(SX_i)]²/[{nSX_i² – (SX_i)²}{nSY_i² – (SY_i)²}]

A convenient formula to compute r² is

r² = {aSY_i - b SX_iY_i – (1/n) (SY_i)²}/{SY_i² - (1/n) (SY_i)²}