prevent for-loop from stopping after an lmer warning/error
It's possible that lmerTest is doing slightly weird things with trying to catch the errors itself. Rune Haubo is working on a revamped version of lmerTest (https://rdrr.io/github/runehaubo/lmerTestR/); it *might* be worth taking a try at installing that to see if it behaves better? On Fri, Mar 23, 2018 at 5:57 PM, Aleksander Adam Glowka
<aglowka at stanford.edu> wrote:
Hi Ben,
Thank you for pointing this out. Indeed the loops only stops after errors, not warnings.
By the way, I am still stuck with my issue. I've made some progress with using tryCatch() as suggested, but still have not resolved my issue. As far as I can tell, currently there are two issues with my code. First, when I query the returned exception in `tryCatch()`, e.g. `poss.err.mod1` I get the following error:
<simpleError in tryCatchList(expr, classes, parentenv, handlers): argument "expr" is missing, with no default>
Still, the warnings are correctly written to file, which is puzzling. Second, when no error is detected, the code seems to be ignored (i.e. no model summaries are written to file). So the loop is running vacuously, skipping errors but apparently handling them and doing nothing else. If anyone has any advice on how to debug this, please let me know!
# divert console stream to file
options(warn=1)
wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
sink(wngs,type="message")
for(iter in 1:1000){
data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
#error handling case #1
poss.err.mod1 = tryCatch(
lmer(y ~ x1 + (1|ranef), data = data, REML = FALSE), #try part
error=function(e) e #catch part
)
#error handling case #2
poss.err.mod2 = tryCatch(
lmer(y ~ x1 + x2 + (1|ranef), data = data, REML = FALSE), #try part
error=function(e) e #catch part
)
#real work #1
if(!inherits(poss.err.mod1, "error")){
mod1 = lmer(y ~ x1 + (1|ranef1), data = data, REML = FALSE)
write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv", sep="")
}else{
write(unlist(mod1 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
}
#real work #2
if(!inherits(poss.err.mod2, "error")){
mod2 = lmer(y ~ x1 + x2 + (1|ranef1), data = data, REML = FALSE)
write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv",)
}else{
write(unlist(mod2 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
}
cat("Iteration", iter, "completed!\n")
}
#close log file & restore warnings stream to console
closeAllConnections()
________________________________
From: Ben Bolker <bbolker at gmail.com>
Sent: Friday, March 23, 2018 10:42:47 AM
To: Andrew Robinson
Cc: Aleksander Adam Glowka; Help Mixed Models
Subject: Re: [R-sig-ME] prevent for-loop from stopping after an lmer warning/error
Is your loop really stopping after warnings (that would be
surprising), not just after errors?
On Thu, Mar 22, 2018 at 3:42 PM, Andrew Robinson <apro at unimelb.edu.au> wrote:
?try
?tryCatch
Cheers,
Andrew
On 23 March 2018 at 06:18, Aleksander Adam Glowka <aglowka at stanford.edu>
wrote:
Hi all,
I'm fitting mixed-effects regression models to bootstrap samples in a
for-loop and writing a subset of model results to file. I diverted the
stream from the console to a text file so I can keep track any warnings or
errors. For some samples the model does not converge and a warning is
issued. For other samples there is an error in calculation of the
Satterthwaite's approximation. The problem is that my loop iteration is
aborted for some reason after a warning is issued. Do you know why this
might be happening and how I can make the loops continue to the next
iteration after an error or a warning?
Below I've included an abridged version of my script and the warnings and
errors I get. I'd be grateful for any advice you may have!
Thank you,
Aleksander Glowka
PhD Candidate
Department of Linguistics
Stanford University
#packages
require(lme4)
require(lmerTest)
setwd(path0)
source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
# divert messages stream to file, so you can log warnings and errors
options(warn=1)
wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
sink(wngs,type="message")
for(iter in 1:2000){
setwd(path1)
data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
#log model number to file so that any potential warnings/errors appear
underneath
message(paste("holistic model #", iter, sep=""))
mod = lmer(y ~ x1 +
x2 +
x3 +
(1|ranef1) +
(x1|ranef2),
data = data,
REML = FALSE)
#write model results to file
setwd(path2)
write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
".csv", sep=""), row.names=TRUE) #fixed effects
write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
write.csv(as.data.frame(holistic.mod at optinfo$derivs$Hessian),
paste("mod.hessian_", iter, ".csv", sep=""))
write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
".csv", sep=""), row.names=FALSE)
write(unlist(holistic.mod at optinfo$conv$lme4$messages),
paste("mod.warnings_", iter, ".txt", sep=""))
cat("Iteration", iter, "completed!\n")
}
#close log file & restore warnings stream to console
closeAllConnections()
Errors:
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
control$checkConv, :
unable to evaluate scaled gradient
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
control$checkConv, :
Model failed to converge: degenerate Hessian with 1 negative eigenvalues
Error in calculation of the Satterthwaite's approximation. The output of
lme4 package is returned
summary from lme4 is returned
some computational error has occurred in lmerTest
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Andrew Robinson
Director, CEBRA, School of BioSciences
Reader & Associate Professor in Applied Statistics Tel: (+61) 0403 138 955
School of Mathematics and Statistics Fax: (+61) 03
8344 4599
University of Melbourne, VIC 3010 Australia
Email: apro at unimelb.edu.au
Website: http://www.ms.unimelb.edu.au/~andrewpr
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models