How to use all the cores while running glmer on a piecewise exponential survival with

Thanks! We are looking into our options. The MixedModels package in Julia
benchmarks at about 2 orders of magnitude faster than R on a small dataset;
however, I would think a lot of that is just overhead from R. On a model of
this size, the computational time should converge because everyone is using
the same BLAS libraries. It might be worth further investigation if timing
remains an issue.

On Thu, Aug 23, 2018 at 2:27 PM D. Rizopoulos <d.rizopoulos at erasmusmc.nl>
wrote:

As suggested, an approach could be to split the original big sample in
manageable pieces, do the analysis in each, and then combine the results.

Geert Molenberghs, Geert Verbeke and colleagues have worked on this; a
relevant recent papers seems to be:
https://lirias2repo.kuleuven.be/bitstream/id/470902/

I hope it helps.

Best,
Dimitris


From: Ben Bolker <bbolker at gmail.com<mailto:bbolker at gmail.com>>
Date: Thursday, 23 Aug 2018, 10:29 PM
To: r-sig-mixed-models at r-project.org <r-sig-mixed-models at r-project.org
<mailto:r-sig-mixed-models at r-project.org>>
Subject: Re: [R-sig-ME] How to use all the cores while running glmer on a
piecewise exponential survival with


  Are the frequentist methods *not* faster?  I'd be pretty surprised,
unless some you're hitting some terrible memory bottleneck or something.


On 2018-08-23 03:30 PM, Adam Mills-Campisi wrote:

We originally tried to use stan to estimate the model, we were getting
performance issues. I assumed that the frequentist approaches would be
faster.

On Thu, Aug 23, 2018 at 12:28 PM Doran, Harold <HDoran at air.org> wrote:

No. You can change to an improved BLAS or I have found the Microsoft R

has

some built in multithreading that is fast for matrix algebra and it

passes

that benefit to lmer. From some experience, you can improve

computational

time of an lmer model with Microsoft R

-----Original Message-----
From: R-sig-mixed-models <r-sig-mixed-models-bounces at r-project.org>

On

Behalf Of Adam Mills-Campisi
Sent: Thursday, August 23, 2018 3:18 PM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] How to use all the cores while running glmer on a
piecewise exponential survival with

I am estimating a piecewise exponential, mixed-effects, survival model
with recurrent events. Each individual in the dataset gets an

individual

interpret (where using a PWP approach). Our full dataset has 10

million

individuals, with 180 million events. I am not sure that there is any
framework which can accommodate data at that size, so we are going to
sample. Our final sample size largely depends on how quickly we can
estimate the model, which brings me to my question: Is there a way to
mutli-thread/core the model? I tried to find some kind of instruction

on

the web and the best lead I could find was a reference to this list

serve.

Any help would be greatly appreciated.

        [[alternative HTML version deleted]]

       [[alternative HTML version deleted]]

        [[alternative HTML version deleted]]