Skip to content

Running

2 messages · Jonathan Miller, Ben Bolker

Original
Jonathan Miller
# 
Dr. Bolker,

I am a Phd student at NCSU and struggling with a coding issue. I am
bootstrapping some glmm model predictions in order to determine the
uncertainty associated with their fixed effects.  I read your comments on
https://github.com/lme4/lme4/issues/388 and have used a code similar to
yours below (b3):

## param, RE, and conditional
b1 <- bootMer(fm1,FUN=sfun1,nsim=100,seed=101)
## param and RE (no conditional)
b2 <- bootMer(fm1,FUN=sfun2,nsim=100,seed=101)
## param only
b3 <- bootMer(fm1,FUN=function(x) predict(x,newdata=test,re.form=~0),
              ## re.form=~0 is equivalent to use.u=FALSE
              nsim=100,seed=101)


It has worked well for me but takes an extremely long time to run. I am
predicting 6 different wq indicators for 1,423 sites and the datasets range
in size from 3,000 to 25,000 entries each.  The small one is relatively
runs relatively ok, but the others are extremely slow. In my code (below),
I also want to make more than one prediction (current conditions, possible
future conditions) using the bootstrapping. Using "snow" in parallel
doesn't seem to speed things up.  I thought of two possibilities, but am
unsure how to implement them.

for (s in 1:1423){

bi <- bootMer(BI.mod,FUN=function(x)
predict(x,newdata=pred.sites[s,],re.form=~0,REML=TRUE),
              parallel="snow",nsim=1000,seed=101)
bi.5 <- bootMer(BI.mod,FUN=function(x)
predict(x,newdata=pred.sites.m5[s,],re.form=~0,REML=TRUE),
              parallel="snow",nsim=1000,seed=101)
}

1) Can I predict the bootstrapped model using two different datasets at
once to speed things up (i.e., pred.sites and pred.sites.m5)?
2) Can I use parallel processing of the initial loop (1,423 sites) outside
of bootmer (perhaps with doParallel and foreach) and then run bootmer
within that loop?  Though I have used foreach before, I find it hard to
compile the data that I really want on the backend.

Thank you for your time and any suggestions you might have.

Sincerely,

Jonathan
Ben Bolker
# 
I will give this some thought when I get a chance (hopefully someone
else will give it some thought and find some answers sooner ...)  In the
meantime -- do you really need parametric bootstrapping/bootMer to get
the confidence intervals you want?  It's quite possible that a simpler
approximation (e.g. assuming that the variation caused by uncertainty in
the top-level random-effects parameters is small relative to other
sources of variability) is adequate, given that you have thousands of
samples ...
On 2018-11-09 4:15 p.m., Jonathan Miller wrote: