Skip to content

Implementing step-wise linear regression

4 messages · Troy S, Tal Galili, Ben Bolker +1 more

Ben Bolker
# 
Tal Galili <tal.galili <at> gmail.com> writes:
[snip snip]
var.test isn't what you want at all; it is for comparing variances
among *populations*.
Not completely (?step uses AIC).
  However add1(...,test="F") is probably what you're looking for.

  Please note that stepwise regression is *strongly* deprecated
by many statisticians, e.g.

<http://www.stata.com/support/faqs/stat/stepwise.html>

Harrell, Frank. 2001. Regression Modeling Strategies. Springer.

Mundry, Roger, and Charles L. Nunn. 2009. Stepwise Model Fitting and Statistical
Inference: Turning Noise into Signal Pollution. The American Naturalist 173, no.
1 (January 1): 119-123. doi:10.1086/593303.
http://www.journals.uchicago.edu/doi/abs/10.1086/593303.

WHITTINGHAM, MARK J., Philip A. Stephens, Richard B. Bradbury, and Robert P.
Freckleton. 2006. Why do we still use stepwise modelling in ecology and
behaviour? Journal of Animal Ecology 75, no. 5: 1182-1189.
Bert Gunter
# 
FWIW, I think it fair to say that modern statistical practice
generally views stepwise regression as a bad idea, especially in the
hands of non-experts lke yourself. The procedures you describe are
"dangerous": they have an uncomfortably high chance of choosing the
wrong variables and leading to widely overoptimistic assessments of
the predictive value of the variables that are chosen. This leads to
scientifically irreproducible results, otherwise known as nonsense (in
polite company; I use another impolite term when I am not being nice).

Shrinkage in its various manifestations is a much better way to
achieve parsimony. See, e.g. the elasticnet, glmnet, pspline, mgcv,
penalized, ... R packages and the MachineLearning task view on CRAN
for various approaches and implementations. Better yet, consult a
local, knowledgeable statistician to help you with this.

Cheers,
Bert
On Mon, Jan 24, 2011 at 12:03 AM, Tal Galili <tal.galili at gmail.com> wrote: