Skip to content

Where to put tryCatch or similar in a very big for loop

7 messages · Steve Lianoglou, Kenneth Takagi, Bonnett, Laura +1 more

Original
Bonnett, Laura
# 
Dear all,

I am running a simulation study to test variable imputation methods for Cox models using R 2.9.0 and Windows XP.  The code I have written (which is rather long) works (if I set nsim = 9) with the following starting values.
I need to run the code 1400 times in total (bootstrap resampling) however, occasionally the random numbers generated lead to a singularity and hence the code crashes as one of the Cox model cannot be fitted (the 10th iteration is the first time this happens).

I've been trawling the internet for ideas and it seems that there are several options in the form of try() or tryCatch() or next.  I'm not sure however, how to include them in my code (attached).  Ideally I'd like it to run everything simulation from 1 to 1400 and if there is an error at some point get an error message returned (I need to count how many there are) but move onto the next number in the loop.

I've tried putting try(....,silent=TRUE) around each cox model (cph statement) but that hasn't work and I've also tried putting try around the whole for loop without any success.

Many thanks for any suggestions you can offer.
Laura
Steve Lianoglou
# 
Hi Laura,

On Thu, Sep 15, 2011 at 10:53 AM, Bonnett, Laura
<L.J.Bonnett at liverpool.ac.uk> wrote:
Let's imagine you are using an `lapply` instead of `for`, only because
I guess you want to store the results of `bootrs` somewhere, you can
adapt this to your `for` solution. I typically return NULL when an
error is caught, then filter those out from my results, or whatever
you like:

results <- lapply(1:1400, function(i) {
  tryCatch(bootrs(...whatever...), error=function(e) NULL)
})
went.south <- sapply(results, is.null)

The `went.south` vector will be TRUE where an error occurred in your
bootrs call.

HTH,
-steve
Bonnett, Laura
# 
Hi Steve,

Thanks for your response.  The slight issue is that I need to use a different starting seed for each simulation.  If I use 'lapply' then I end up using the same seed each time.  (By contrast, I need to be able to specify which starting seed I am using).

Thanks,
Laura

-----Original Message-----
From: Steve Lianoglou [mailto:mailinglist.honeypot at gmail.com] 
Sent: 15 September 2011 16:17
To: Bonnett, Laura
Cc: r-help at r-project.org
Subject: Re: [R] Where to put tryCatch or similar in a very big for loop

Hi Laura,

On Thu, Sep 15, 2011 at 10:53 AM, Bonnett, Laura
<L.J.Bonnett at liverpool.ac.uk> wrote:
Let's imagine you are using an `lapply` instead of `for`, only because
I guess you want to store the results of `bootrs` somewhere, you can
adapt this to your `for` solution. I typically return NULL when an
error is caught, then filter those out from my results, or whatever
you like:

results <- lapply(1:1400, function(i) {
  tryCatch(bootrs(...whatever...), error=function(e) NULL)
})
went.south <- sapply(results, is.null)

The `went.south` vector will be TRUE where an error occurred in your
bootrs call.

HTH,
-steve
Kenneth Takagi
# 
What type of singularity exactly, if you're working with counts is it a special case? If using a Monte Carlo generation scheme, there are various workarounds such as while(sum(vec)!=0) {sample} for example. More info on the error circumstances would help.
   Good luck!
    Ken Hutchison
On Sep 15, 2554 BE, at 11:41 AM, "Bonnett, Laura" <L.J.Bonnett at liverpool.ac.uk> wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Bonnett, Laura
    

    

      
      #
    

  


  

      
Hi,

The simulation occasionally generates either a rare event meaning that the Cox model is not appropriate or it generates a covariate with most responses being the same which means that the Cox model cannot be fit.

At bootstrap sample number 10, the variable c11 is considered singular by model cox1.

Thanks,
Laura

-----Original Message-----
From: Ken [mailto:vicvoncastle at gmail.com] 
Sent: 15 September 2011 21:43
To: Bonnett, Laura
Cc: Steve Lianoglou; r-help at r-project.org
Subject: Re: [R] Where to put tryCatch or similar in a very big for loop

What type of singularity exactly, if you're working with counts is it a special case? If using a Monte Carlo generation scheme, there are various workarounds such as while(sum(vec)!=0) {sample} for example. More info on the error circumstances would help.

   Good luck!
    Ken Hutchison

        
On Sep 15, 2554 BE, at 11:41 AM, "Bonnett, Laura" <L.J.Bonnett at liverpool.ac.uk> wrote:

        

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Jan van der Laan
    

    

      
      #
    

  


  

      
Laura,

Perhaps the following example helps:

nbstr <- 100
result <- numeric(nbstr)
for (i in seq_len(nbstr)) {
   # set the default value for when the current bootstrap fails
   result[i] <- NA
   try({
     # estimate your cox model here
     if (runif(1) < 0.1) stop("ERROR")
     result[i] <- i
   }, silent=TRUE)
}

Regards,
Jan




Quoting "Bonnett, Laura" <L.J.Bonnett at liverpool.ac.uk>:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Bonnett, Laura
    

    

      
      #
    

  


  

      
Hi all,

I've made some progress today.  I've managed to move the set seed part of the function outside of the loop.  The only problem I have now is that it says 

" Error in results[k, ] <- lapply(1:nsim, function(k) { : 
  number of items to replace is not a multiple of replacement length"

Given I'm using the code attached can anyone offer a solution?

Many thanks,
Laura

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Bonnett, Laura
Sent: 16 September 2011 08:22
To: 'Ken'
Cc: r-help at r-project.org
Subject: Re: [R] Where to put tryCatch or similar in a very big for loop

Hi,

The simulation occasionally generates either a rare event meaning that the Cox model is not appropriate or it generates a covariate with most responses being the same which means that the Cox model cannot be fit.

At bootstrap sample number 10, the variable c11 is considered singular by model cox1.

Thanks,
Laura

-----Original Message-----
From: Ken [mailto:vicvoncastle at gmail.com] 
Sent: 15 September 2011 21:43
To: Bonnett, Laura
Cc: Steve Lianoglou; r-help at r-project.org
Subject: Re: [R] Where to put tryCatch or similar in a very big for loop

What type of singularity exactly, if you're working with counts is it a special case? If using a Monte Carlo generation scheme, there are various workarounds such as while(sum(vec)!=0) {sample} for example. More info on the error circumstances would help.


   Good luck!
    Ken Hutchison

        
On Sep 15, 2554 BE, at 11:41 AM, "Bonnett, Laura" <L.J.Bonnett at liverpool.ac.uk> wrote:

        

            

      
      
      
    

        

      
      
    

        
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.