Skip to content
Prev 14669 / 20628 Next
Original Raw

afex package & bootstrapping

Henrik Singmann 
Hi Dan,

Your code looks reasonable to me. details = 2 is probably responsible 
for the following message which shows you the time it took to calculate 
the reference distribution for the first p-value:
"Reference distribution with   996 samples; computing time: 159891.87 secs."

The one period in the output also shows you that the first of 15 
p-values has been successfully calculated and the next is being calculating.

So it seems that calculating the reference distribution for the first 
p-values took around 44 hours on your 12 cores.

Unfortunately, the speed up by using parallel computation is not linear 
when calculating the reference distribution using pbkrtest::PBmodcomp 
(which mixed uses as for the computation). This is most likely due to 
the fact that some models sometimes are faster than others and then the 
cores idle before starting with the next model until all cores can start 
the next model (I didn't look at the code of PBmodcomp, but that seems 
somewhat likely).

On my 8 cores one can nevertheless see a considerable speed up when 
using multicore compared to when not, using an example model (16 samples 
is obviously a stupid example):

require(afex)
require("mlmRev")
require(parallel)
(nc <- detectCores()) # 8 in my case
cl <- makeCluster(rep("localhost", nc))

mixed(use ~ age + I(age^2) + urban + livch + (1 | district), family = 
binomial, method = "PB", data = Contraception, args.test = list(nsim = 
16, cl = cl, details = 2))

## Contrasts set to contr.sum for the following variables: use, urban, 
livch, district
## Numerical variables NOT centered on 0 (i.e., interpretation of all 
main effects might be difficult if in interactions): age
## Fitting 5 (g)lmer() models:
## [.....]
## Obtaining 4 p-values:
## [Reference distribution with    16 samples; computing time: 33.29 secs.
## .Reference distribution with    16 samples; computing time: 37.25 secs.
## .Reference distribution with    16 samples; computing time: 41.06 secs.
## .Reference distribution with    16 samples; computing time: 27.92 secs.
## .]

mixed(use ~ age + I(age^2) + urban + livch + (1 | district), family = 
binomial, method = "PB", data = Contraception, args.test = list(nsim = 
16, details = 2))
## Contrasts set to contr.sum for the following variables: use, urban, 
livch, district
## Numerical variables NOT centered on 0 (i.e., interpretation of all 
main effects might be difficult if in interactions): age
## Fitting 5 (g)lmer() models:
## [.....]
## Obtaining 4 p-values:
## [Reference distribution with    16 samples; computing time: 143.49 secs.
## .Reference distribution with    16 samples; computing time: 201.51 secs.
## .Reference distribution with    16 samples; computing time: 185.76 secs.
## .Reference distribution with    16 samples; computing time: 120.80 secs.
## .]


You can see a speed up from around 150 to 35 seconds per p-value which 
corresponds to around a fourfold increase in speed, but not anywhere 
near an eightfold increase. So you cannot do much other than wait if you 
want to use parametric bootstrap and not e.g., "LRT" (or use a machine 
with more cores, but given the assumed problem, the speed up will most 
likely be less and less).

Hope that helps,
Henrik  (author of afex)



Am 05.07.2016 um 22:59 schrieb Daniel McCloy:

Thread (2 messages)

Dan McCloy afex package & bootstrapping Jul 5 Henrik Singmann afex package & bootstrapping Jul 6