How to use all the cores while running glmer on a piecewise exponential survival with
As suggested, an approach could be to split the original big sample in manageable pieces, do the analysis in each, and then combine the results. Geert Molenberghs, Geert Verbeke and colleagues have worked on this; a relevant recent papers seems to be: https://lirias2repo.kuleuven.be/bitstream/id/470902/ I hope it helps. Best, Dimitris From: Ben Bolker <bbolker at gmail.com<mailto:bbolker at gmail.com>> Date: Thursday, 23 Aug 2018, 10:29 PM To: r-sig-mixed-models at r-project.org <r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>> Subject: Re: [R-sig-ME] How to use all the cores while running glmer on a piecewise exponential survival with Are the frequentist methods *not* faster? I'd be pretty surprised, unless some you're hitting some terrible memory bottleneck or something.
On 2018-08-23 03:30 PM, Adam Mills-Campisi wrote:
We originally tried to use stan to estimate the model, we were getting performance issues. I assumed that the frequentist approaches would be faster. On Thu, Aug 23, 2018 at 12:28 PM Doran, Harold <HDoran at air.org> wrote:
No. You can change to an improved BLAS or I have found the Microsoft R has
some built in multithreading that is fast for matrix algebra and it passes
that benefit to lmer. From some experience, you can improve computational
time of an lmer model with Microsoft R
-----Original Message-----
From: R-sig-mixed-models <r-sig-mixed-models-bounces at r-project.org> On
Behalf Of Adam Mills-Campisi
Sent: Thursday, August 23, 2018 3:18 PM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] How to use all the cores while running glmer on a
piecewise exponential survival with
I am estimating a piecewise exponential, mixed-effects, survival model
with recurrent events. Each individual in the dataset gets an individual
interpret (where using a PWP approach). Our full dataset has 10 million
individuals, with 180 million events. I am not sure that there is any
framework which can accommodate data at that size, so we are going to
sample. Our final sample size largely depends on how quickly we can
estimate the model, which brings me to my question: Is there a way to
mutli-thread/core the model? I tried to find some kind of instruction on
the web and the best lead I could find was a reference to this list serve.
Any help would be greatly appreciated.
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models