ridge regression

Hi Frank,
From: Frank E Harrell Jr [mailto:fharrell at virginia.edu]
[snip]
The anova method for ols fits 'works' when you penalize the 
model but there is some controversy over whether we should be 
testing biased coefficients.  Some believe that hypothesis 
tests should be done using the unpenalized model.  That 
brings up other ways to handle collinearity: test groups of 
variables in combination so they don't compete with each 
other, or collapse them into summary scores (e.g., principal 
components) before putting them in the model. 
I'm not clear about the last point.  Suppose three of the variables
are nearly collinear.  Are you suggesting to replace the variables
with the first one or two PCs, and drop the rest?  If so, doesn't
that also lead to biased estimators?

Best,
Andy
---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  
http://hesweb1.med.virginia.edu/biostat
------------------------------------------------------------------------------
Notice: This e-mail message, together with any attachments, cont... {{dropped}}
On Thu, 05 Jun 2003 21:50:13 -0400

Hi Frank,

From: Frank E Harrell Jr [mailto:fharrell at virginia.edu]
[snip]

The anova method for ols fits 'works' when you penalize the 
model but there is some controversy over whether we should be 
testing biased coefficients.  Some believe that hypothesis 
tests should be done using the unpenalized model.  That 
brings up other ways to handle collinearity: test groups of 
variables in combination so they don't compete with each 
other, or collapse them into summary scores (e.g., principal 
components) before putting them in the model. 
I'm not clear about the last point.  Suppose three of the variables
are nearly collinear.  Are you suggesting to replace the variables
with the first one or two PCs, and drop the rest?  If so, doesn't
that also lead to biased estimators?
Yes, it's another way to get biased estimators.  But many statisticians test these components in incomplete principal components regression using ordinary tests, to get tests for the whole group of coefficients these represent (under restrictions).  -Frank
Best,
Andy

---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  
http://hesweb1.med.virginia.edu/biostat
------------------------------------------------------------------------------
Notice: This e-mail message, together with any attachments, cont... {{dropped}}

______________________________________________
R-help at stat.math.ethz.ch mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat