ridge regression
On Thu, 05 Jun 2003 21:50:13 -0400
"Liaw, Andy" <andy_liaw at merck.com> wrote:
Hi Frank,
From: Frank E Harrell Jr [mailto:fharrell at virginia.edu]
[snip]
The anova method for ols fits 'works' when you penalize the model but there is some controversy over whether we should be testing biased coefficients. Some believe that hypothesis tests should be done using the unpenalized model. That brings up other ways to handle collinearity: test groups of variables in combination so they don't compete with each other, or collapse them into summary scores (e.g., principal components) before putting them in the model.
I'm not clear about the last point. Suppose three of the variables are nearly collinear. Are you suggesting to replace the variables with the first one or two PCs, and drop the rest? If so, doesn't that also lead to biased estimators?
Yes, it's another way to get biased estimators. But many statisticians test these components in incomplete principal components regression using ordinary tests, to get tests for the whole group of coefficients these represent (under restrictions). -Frank
Best, Andy
--- Frank E Harrell Jr Prof. of Biostatistics & Statistics Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences U. Virginia School of Medicine http://hesweb1.med.virginia.edu/biostat
------------------------------------------------------------------------------
Notice: This e-mail message, together with any attachments, cont... {{dropped}}
______________________________________________ R-help at stat.math.ethz.ch mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
--- Frank E Harrell Jr Prof. of Biostatistics & Statistics Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences U. Virginia School of Medicine http://hesweb1.med.virginia.edu/biostat