×

A note on prediction error with collinearity. (English) Zbl 1062.62119

Summary: The least squares (LS) estimator is the best linear unbiased estimator for linear models. It is well known that the LS performs poorly in estimation when collinearity is present among regressors. However, it is not fully understood and is even controversial whether LS performs well in prediction. To address this controversy, we study the mean and variance of the prediction squared error (PSE) of the LS estimator, and conclude theoretically that although the mean PSE remains invariant regardless of the collinearity, the variance of PSE increases with the collinearity. Thus the prediction error is sensitive to the location in the feature space.

MSC:

62J05 Linear regression; mixed models
62H12 Estimation in multivariate analysis
62J07 Ridge regression; shrinkage estimators (Lasso)
Full Text: DOI

References:

[1] DOI: 10.1198/016214501753382273 · Zbl 1073.62547 · doi:10.1198/016214501753382273
[2] DOI: 10.1214/aos/1015362185 · Zbl 1012.62106 · doi:10.1214/aos/1015362185
[3] DOI: 10.2307/1390712 · doi:10.2307/1390712
[4] DOI: 10.1080/03610920008832483 · Zbl 1026.62074 · doi:10.1080/03610920008832483
[5] DOI: 10.1111/1541-0420.00015 · Zbl 1210.62016 · doi:10.1111/1541-0420.00015
[6] DOI: 10.2307/1269656 · Zbl 0775.62288 · doi:10.2307/1269656
[7] George E. I., JASA 95 pp 1304– (2000)
[8] Gruber M. H. J., Regression Estimators: A Comparative Study. (1990) · Zbl 0705.62063
[9] Hastie T., Generalized Additive Models. (1990) · Zbl 0747.62061
[10] DOI: 10.2307/1267351 · Zbl 0202.17205 · doi:10.2307/1267351
[11] DOI: 10.2307/1267352 · Zbl 0202.17206 · doi:10.2307/1267352
[12] DOI: 10.1214/aos/1015957397 · Zbl 1105.62357 · doi:10.1214/aos/1015957397
[13] DOI: 10.1016/S0167-7152(02)00140-2 · Zbl 1092.62570 · doi:10.1016/S0167-7152(02)00140-2
[14] Miller A. J., Subset Selection in Regression. (1990) · Zbl 0702.62057
[15] Sen A., Regression Analysis: Theory, Methods, and Applications. (1990) · Zbl 0714.62057
[16] DOI: 10.1007/978-1-4612-0795-5 · doi:10.1007/978-1-4612-0795-5
[17] Tibshirani R., J. Roy. Statist. Soc. B 58 pp 267– (1996)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.