Leave one out Cross validation (LOO)
Alex Roy wrote:
Dear Frank,
Thanks for your comments. But in my situation, I do
not have any future data and I want to calculate Mean Square Error for
prediction on future data. So, is it not it a good idea to go for LOO?
thanks
Alex
With resampling you should be able to estimate any parameter including sigma. The Design package's validate.ols function can estimate sigma using the bootstrap or c-v, penalizing for backward stepdown variable selection, although I have found some counter-intuitive estimates of sigma using Efron's optimism bootstrap. Frank
On Tue, Feb 24, 2009 at 7:15 PM, Frank E Harrell Jr
<f.harrell at vanderbilt.edu <mailto:f.harrell at vanderbilt.edu>> wrote:
Alex Roy wrote:
Dear R user,
I am working with LOO. Can any one who is
working
with leave one out cross validation (LOO) could send me the code?
Thanks in advance
Alex
I don't think that LOO adequately penalizes for model uncertainty.
I recommend the bootstrap or 50 repeats of 10-fold
cross-validation. See for example the validate and calibrate
functions in the R Design package.
Frank
--