prevent for-loop from stopping after an lmer warning/error

?try

?tryCatch

Cheers,

Andrew


On 23 March 2018 at 06:18, Aleksander Adam Glowka <aglowka at stanford.edu>
wrote:

Hi all,

I'm fitting mixed-effects regression models to bootstrap samples in a
for-loop and writing a subset of model results to file. I diverted the
stream from the console to a text file so I can keep track any warnings or
errors. For some samples the model does not converge and a warning is
issued. For other samples there is an error in calculation of the
Satterthwaite's approximation. The problem is that my loop iteration is
aborted for some reason after a warning is issued. Do you know why this
might be happening and how I can make the loops continue to the next
iteration after an error or a warning?

Below I've included an abridged version of my script and the warnings and
errors I get. I'd be grateful for any advice you may have!

Thank you,

Aleksander Glowka
PhD Candidate
Department of Linguistics
Stanford University

#packages
require(lme4)
require(lmerTest)

setwd(path0)
source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor

# divert messages stream to file, so you can log warnings and errors
options(warn=1)
wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
sink(wngs,type="message")

for(iter in 1:2000){

  setwd(path1)

  data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)

  #log model number to file so that any potential warnings/errors appear
underneath
  message(paste("holistic model #", iter, sep=""))

  mod = lmer(y ~ x1 +
            x2 +
            x3 +
            (1|ranef1) +
            (x1|ranef2),
            data = data,
            REML = FALSE)

  #write model results to file
  setwd(path2)
  write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
".csv", sep=""), row.names=TRUE) #fixed effects
  write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
  write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
  write.csv(as.data.frame(holistic.mod at optinfo$derivs$Hessian),
paste("mod.hessian_", iter, ".csv", sep=""))
  write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
".csv", sep=""), row.names=FALSE)
  write(unlist(holistic.mod at optinfo$conv$lme4$messages),
paste("mod.warnings_", iter, ".txt", sep=""))

  cat("Iteration", iter, "completed!\n")

}

#close log file & restore warnings stream to console
closeAllConnections()

Errors:

Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
control$checkConv,  :
  unable to evaluate scaled gradient
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues

Error in calculation of the Satterthwaite's approximation. The output of
lme4 package is returned
summary from lme4 is returned
some computational error has occurred in lmerTest




        [[alternative HTML version deleted]]

--
Andrew Robinson
Director, CEBRA, School of BioSciences
Reader & Associate Professor in Applied Statistics  Tel: (+61) 0403 138 955
School of Mathematics and Statistics                        Fax: (+61) 03
8344 4599
University of Melbourne, VIC 3010 Australia
Email: apro at unimelb.edu.au
Website: http://www.ms.unimelb.edu.au/~andrewpr

        [[alternative HTML version deleted]]

Hi Ben,


Thank you for pointing this out. Indeed the loops only stops after errors, not warnings.


By the way, I am still stuck with my issue. I've made some progress with using tryCatch() as suggested, but still have not resolved my issue. As far as I can tell, currently there are two issues with my code. First, when I query the returned exception in `tryCatch()`, e.g. `poss.err.mod1` I get the following error:

    <simpleError in tryCatchList(expr, classes, parentenv, handlers): argument "expr" is missing, with no default>

Still, the warnings are correctly written to file, which is puzzling. Second, when no error is detected, the code seems to be ignored (i.e. no model summaries are written to file). So the loop is running vacuously, skipping errors but apparently handling them and doing nothing else. If anyone has any advice on how to debug this, please let me know!

    # divert console stream to file
    options(warn=1)
    wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
    sink(wngs,type="message")

    for(iter in 1:1000){

      data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)

      #error handling case #1
      poss.err.mod1 = tryCatch(

        lmer(y ~ x1 + (1|ranef), data = data, REML = FALSE), #try part
        error=function(e) e #catch part
      )

      #error handling case #2
      poss.err.mod2 = tryCatch(

        lmer(y ~ x1 + x2 + (1|ranef), data = data, REML = FALSE), #try part
        error=function(e) e #catch part
      )

      #real work #1
      if(!inherits(poss.err.mod1, "error")){

        mod1 = lmer(y ~ x1 + (1|ranef1), data = data, REML = FALSE)
        write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv", sep="")

      }else{

        write(unlist(mod1 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
      }

      #real work #2
      if(!inherits(poss.err.mod2, "error")){

        mod2 = lmer(y ~ x1 + x2 + (1|ranef1), data = data, REML = FALSE)
        write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv",)

      }else{

        write(unlist(mod2 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
      }

      cat("Iteration", iter, "completed!\n")
    }

    #close log file & restore warnings stream to console
    closeAllConnections()