Dear members,
I am trying to set some code running on Windows, which calculates and
returns the Laplace-based maximum likelihood estimates of a generalized
linear mixed model.
Both glmer and glmmTMB functions work on R 3.3.3 when the code is executed
sequentially.
However, when I use the parallel algorithm, then glmer still works, but
glmmTMB does not.
The error I get is Error in { : task 1 failed - "object 'dat' not found" ,
where dat is the simulated dataset in each replication.
Any ideas on how to tackle this error? I am interested in using glmmTMB
instead of lme4, because I need some of glmmTMB output components.
Below is a sample code that illustrates the above:
library(doParallel)
library(glmmTMB)
library(lme4)
simSamples <- matrix(sample(0:6,300,replace=TRUE),30,10)
## Sequential version
LA <- LA2 <- matrix(NA,3,10)
for (i in 1:10){
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),y=simSamples[,i],yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,weights=m,data=dat)
LA[,i] <- fitLA$fit$par
fitLA2 <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
LA2[,i] <- c(fixef(fitLA2),sqrt(unlist(VarCorr(fitLA2))))
}
## Parallel version
cl <- makeCluster(3)
registerDoParallel(cl)
TMBfit <- foreach(i=1:10, .combine= cbind, .packages=c("glmmTMB")) %dopar% {
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,weights=m,data=dat)
fitLA$fit$par
}
stopCluster(cl)
cl <- makeCluster(3)
registerDoParallel(cl)
LME4fit <- foreach(i=1:10, .combine= cbind, .packages=c("lme4")) %dopar% {
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),y=simSamples[,i])
fitLA <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
c(fixef(fitLA),sqrt(unlist(VarCorr(fitLA))))
}
stopCluster(cl)
Any help is much appreciated.
Thanks in advance!
Sophia
glmmTMB and foreach parallel adaptor
5 messages · Sophia Kyriakou, João Veríssimo, Philippi, Tom +1 more
Hi Sophia, Not sure why this happens, but using <<- assignment seems to work. (i.e., dat <<- ..., rather than dat <- ...) Jo?o
On Fri, 2017-03-17 at 00:34 +0200, Sophia Kyriakou wrote:
Dear members,
I am trying to set some code running on Windows, which calculates and
returns the Laplace-based maximum likelihood estimates of a generalized
linear mixed model.
Both glmer and glmmTMB functions work on R 3.3.3 when the code is executed
sequentially.
However, when I use the parallel algorithm, then glmer still works, but
glmmTMB does not.
The error I get is Error in { : task 1 failed - "object 'dat' not found" ,
where dat is the simulated dataset in each replication.
Any ideas on how to tackle this error? I am interested in using glmmTMB
instead of lme4, because I need some of glmmTMB output components.
Below is a sample code that illustrates the above:
library(doParallel)
library(glmmTMB)
library(lme4)
simSamples <- matrix(sample(0:6,300,replace=TRUE),30,10)
## Sequential version
LA <- LA2 <- matrix(NA,3,10)
for (i in 1:10){
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),y=simSamples[,i],yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,weights=m,data=dat)
LA[,i] <- fitLA$fit$par
fitLA2 <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
LA2[,i] <- c(fixef(fitLA2),sqrt(unlist(VarCorr(fitLA2))))
}
## Parallel version
cl <- makeCluster(3)
registerDoParallel(cl)
TMBfit <- foreach(i=1:10, .combine= cbind, .packages=c("glmmTMB")) %dopar% {
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,weights=m,data=dat)
fitLA$fit$par
}
stopCluster(cl)
cl <- makeCluster(3)
registerDoParallel(cl)
LME4fit <- foreach(i=1:10, .combine= cbind, .packages=c("lme4")) %dopar% {
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),y=simSamples[,i])
fitLA <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
c(fixef(fitLA),sqrt(unlist(VarCorr(fitLA))))
}
stopCluster(cl)
Any help is much appreciated.
Thanks in advance!
Sophia
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Thanks Jo?o, indeed the scoping assignment works. All the best, Sophia On Fri, Mar 17, 2017 at 1:09 AM, Jo?o Ver?ssimo <jl.verissimo at gmail.com> wrote:
Hi Sophia, Not sure why this happens, but using <<- assignment seems to work. (i.e., dat <<- ..., rather than dat <- ...) Jo?o On Fri, 2017-03-17 at 00:34 +0200, Sophia Kyriakou wrote:
Dear members, I am trying to set some code running on Windows, which calculates and returns the Laplace-based maximum likelihood estimates of a generalized linear mixed model. Both glmer and glmmTMB functions work on R 3.3.3 when the code is
executed
sequentially.
However, when I use the parallel algorithm, then glmer still works, but
glmmTMB does not.
The error I get is Error in { : task 1 failed - "object 'dat' not found"
,
where dat is the simulated dataset in each replication.
Any ideas on how to tackle this error? I am interested in using glmmTMB
instead of lme4, because I need some of glmmTMB output components.
Below is a sample code that illustrates the above:
library(doParallel)
library(glmmTMB)
library(lme4)
simSamples <- matrix(sample(0:6,300,replace=TRUE),30,10)
## Sequential version
LA <- LA2 <- matrix(NA,3,10)
for (i in 1:10){
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i],yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
LA[,i] <- fitLA$fit$par
fitLA2 <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
LA2[,i] <- c(fixef(fitLA2),sqrt(unlist(VarCorr(fitLA2))))
}
## Parallel version
cl <- makeCluster(3)
registerDoParallel(cl)
TMBfit <- foreach(i=1:10, .combine= cbind, .packages=c("glmmTMB"))
%dopar% {
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
fitLA$fit$par
}
stopCluster(cl)
cl <- makeCluster(3)
registerDoParallel(cl)
LME4fit <- foreach(i=1:10, .combine= cbind, .packages=c("lme4")) %dopar%
{
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i])
fitLA <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
c(fixef(fitLA),sqrt(unlist(VarCorr(fitLA))))
}
stopCluster(cl)
Any help is much appreciated.
Thanks in advance!
Sophia
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
I try to avoid variable scoping (environment) confusion by wrapping my
inner stuff as a simple function:
cl <- makeCluster(3)
registerDoParallel(cl)
fn <- function(i) {
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
return(fitLA$fit$par)
}
TMBfit <- foreach(i=1:10, .combine= cbind, .inorder=FALSE,
.packages=c("glmmTMB")) %dopar% fn(i)
stopCluster(cl)
With .inorder=FALSE you may or may not need to return a list with i as well:
return(list(i=i,par=fitLA$fit$par))
Tom 2
On Thu, Mar 16, 2017 at 4:51 PM, Sophia Kyriakou <
sophia.kyriakou17 at gmail.com> wrote:
Thanks Jo?o, indeed the scoping assignment works. All the best, Sophia On Fri, Mar 17, 2017 at 1:09 AM, Jo?o Ver?ssimo <jl.verissimo at gmail.com> wrote:
Hi Sophia, Not sure why this happens, but using <<- assignment seems to work. (i.e., dat <<- ..., rather than dat <- ...) Jo?o On Fri, 2017-03-17 at 00:34 +0200, Sophia Kyriakou wrote:
Dear members, I am trying to set some code running on Windows, which calculates and returns the Laplace-based maximum likelihood estimates of a generalized linear mixed model. Both glmer and glmmTMB functions work on R 3.3.3 when the code is
executed
sequentially.
However, when I use the parallel algorithm, then glmer still works, but
glmmTMB does not.
The error I get is Error in { : task 1 failed - "object 'dat' not
found"
,
where dat is the simulated dataset in each replication.
Any ideas on how to tackle this error? I am interested in using glmmTMB
instead of lme4, because I need some of glmmTMB output components.
Below is a sample code that illustrates the above:
library(doParallel)
library(glmmTMB)
library(lme4)
simSamples <- matrix(sample(0:6,300,replace=TRUE),30,10)
## Sequential version
LA <- LA2 <- matrix(NA,3,10)
for (i in 1:10){
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i],yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
LA[,i] <- fitLA$fit$par fitLA2 <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,
data=dat)
LA2[,i] <- c(fixef(fitLA2),sqrt(unlist(VarCorr(fitLA2))))
}
## Parallel version
cl <- makeCluster(3)
registerDoParallel(cl)
TMBfit <- foreach(i=1:10, .combine= cbind, .packages=c("glmmTMB"))
%dopar% {
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
fitLA$fit$par
}
stopCluster(cl)
cl <- makeCluster(3)
registerDoParallel(cl)
LME4fit <- foreach(i=1:10, .combine= cbind, .packages=c("lme4"))
%dopar%
{
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i])
fitLA <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
c(fixef(fitLA),sqrt(unlist(VarCorr(fitLA))))
}
stopCluster(cl)
Any help is much appreciated.
Thanks in advance!
Sophia
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Hi Sophia, I think we fixed this bug https://github.com/glmmTMB/glmmTMB/issues/209 <https://github.com/glmmTMB/glmmTMB/issues/209> Have you installed from GitHub in the past 2 weeks? I think this should also be fixed in the CRAN version. If updating from GitHub doesn?t fix the problem, please file an issue or let me know and I will. cheers, Mollie ??????????? Mollie E. Brooks, Ph.D. Postdoctoral Researcher National Institute of Aquatic Resources Technical University of Denmark
On 17Mar 2017, at 1:10, Philippi, Tom <tom_philippi at nps.gov> wrote:
I try to avoid variable scoping (environment) confusion by wrapping my
inner stuff as a simple function:
cl <- makeCluster(3)
registerDoParallel(cl)
fn <- function(i) {
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
return(fitLA$fit$par)
}
TMBfit <- foreach(i=1:10, .combine= cbind, .inorder=FALSE,
.packages=c("glmmTMB")) %dopar% fn(i)
stopCluster(cl)
With .inorder=FALSE you may or may not need to return a list with i as well:
return(list(i=i,par=fitLA$fit$par))
Tom 2
On Thu, Mar 16, 2017 at 4:51 PM, Sophia Kyriakou <
sophia.kyriakou17 at gmail.com> wrote:
Thanks Jo?o, indeed the scoping assignment works. All the best, Sophia On Fri, Mar 17, 2017 at 1:09 AM, Jo?o Ver?ssimo <jl.verissimo at gmail.com> wrote:
Hi Sophia, Not sure why this happens, but using <<- assignment seems to work. (i.e., dat <<- ..., rather than dat <- ...) Jo?o On Fri, 2017-03-17 at 00:34 +0200, Sophia Kyriakou wrote:
Dear members, I am trying to set some code running on Windows, which calculates and returns the Laplace-based maximum likelihood estimates of a generalized linear mixed model. Both glmer and glmmTMB functions work on R 3.3.3 when the code is
executed
sequentially.
However, when I use the parallel algorithm, then glmer still works, but
glmmTMB does not.
The error I get is Error in { : task 1 failed - "object 'dat' not
found"
,
where dat is the simulated dataset in each replication.
Any ideas on how to tackle this error? I am interested in using glmmTMB
instead of lme4, because I need some of glmmTMB output components.
Below is a sample code that illustrates the above:
library(doParallel)
library(glmmTMB)
library(lme4)
simSamples <- matrix(sample(0:6,300,replace=TRUE),30,10)
## Sequential version
LA <- LA2 <- matrix(NA,3,10)
for (i in 1:10){
dat <-
data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i],yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
LA[,i] <- fitLA$fit$par fitLA2 <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,
data=dat)
LA2[,i] <- c(fixef(fitLA2),sqrt(unlist(VarCorr(fitLA2))))
}
## Parallel version
cl <- makeCluster(3)
registerDoParallel(cl)
TMBfit <- foreach(i=1:10, .combine= cbind, .packages=c("glmmTMB"))
%dopar% {
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
yprop=simSamples[,i]/6)
fitLA <- glmmTMB(yprop ~ x +(1|subject),family=binomial,
weights=m,data=dat)
fitLA$fit$par
}
stopCluster(cl)
cl <- makeCluster(3)
registerDoParallel(cl)
LME4fit <- foreach(i=1:10, .combine= cbind, .packages=c("lme4"))
%dopar%
{
dat <- data.frame(subject=rep(1:15,each=2),x=-14:15,m=rep(6,30),
y=simSamples[,i])
fitLA <- glmer(cbind(y,6-y) ~ x +(1|subject),family=binomial,data=dat)
c(fixef(fitLA),sqrt(unlist(VarCorr(fitLA))))
}
stopCluster(cl)
Any help is much appreciated.
Thanks in advance!
Sophia
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models