Skip to content

Using tryCatch in a for loop

9 messages · Bert Gunter, Daniel Nordlund, Bailey Hewitt

#
Hello All, 

I have been trying to use a for loop to run segmented regressions (from?R package segmented)?on many columns  of data in a data frame with the end goal of writing a new file with the following columns: column title, breakpoint year, slope, and difference in slope. Unfortunately, when one of the columns doesn't have a breakpoint the code stops and provides?an error or warning. I would like the loop to keep running regardless of the error/warning but I want it to write that it did encounter an error or warning. Based on my needs I found that tryCatch appears to do what I need. I have looked at multiple examples, blogs, etc. as well as gone over the documentation and Hadley Wickham's document in Advanced R but I am still a novice when it comes to coding and I am having a hard time getting the code to work properly. Below is the code I have developed thus far (with a test dataset as an example):

#Creating a test dataset
Year<- c(2000, 2001, 2002, 2003, 2004)
Lake1<- c(2, 4, 5, 2, 1)
Lake2<- c(1, 3, -1, 4, -2)
Lake3<- c(1, 2, 5, -3, 1)
mydata<- data.frame(Year, Lake1, Lake2, Lake3)

#Running a for loop that indicates when an error or warning occurs
y<- mydata[,2:4]
year <- mydata$Year
regimeshift <- data.frame()
for (i in 1:3){
? tryCatch({
? ? y.val <- y[,i]
? ? lin.reg <- lm(y.val~year, mydata)
? ? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
? ? RSyear <- summary(seg.reg)$psi [1,2]
? ? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]?
? ? SlopeDiff <- summary(seg.reg)$coefficients[3,1]?
? ? new.regimeshift <- data.frame(RSyear=RSyear, SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
? ? rownames(new.regimeshift) <- colnames(y)[i]
? ? regimeshift <- rbind(regimeshift,new.regimeshift)
? ? print(regimeshift)
? }, error= function(e) {cat("Error", "\n")},
? ? warning= function(w) {cat("Warning", "\n")})
}

Any ideas or suggestions you might have are greatly?appreciated!

Bailey Hewitt
PhD Candidate
York University
Ontario, Canada
#
Others may have greater insight, but my response is: Exactly what did or
didn't happen that makes you say the code didn't work? That is, what did or
didn't you get when you ran it compared to your expectations?

Cheers,
Bert



Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Tue, May 22, 2018 at 10:55 AM, Bailey Hewitt <bailster at hotmail.com>
wrote:

  
  
#
Hi Bert,

Thank you for the quick response!?

In its current?state?the code?prints three lines that say "warning". What I was expecting is that I would?get?a matrix with 4 columns, 1. column names (from the original data, ex. Lake1) 2. breakpoint year 3. slope 4. slope difference from the first to the?second segment of the segmented regression. Each row in the matrix would be the results of the segmented regression test for each lake in the original data frame, so?Lake1 results would be in row 2 (row 1 would be titles) and so on. If any of?this is confusing please let me know and I will clarify!

Thanks!

Bailey?




From: Bert Gunter <bgunter.4567 at gmail.com>
Sent: May 22, 2018 2:13 PM
To: Bailey Hewitt
Cc: r-help at R-project.org
Subject: Re: [R] Using tryCatch in a for loop
?  



Others may have greater insight, but my response is: Exactly what did or didn't happen that makes you say the code didn't work? That is, what did or didn't you get when you ran it compared to your expectations?

 Cheers,
 Bert






Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, May 22, 2018 at 10:55 AM, Bailey Hewitt <bailster at hotmail.com> wrote:
Hello All, 

I have been trying to use a for loop to run segmented regressions (from?R package segmented)?on many columns? of data in a data frame with the end goal of writing a new file with the following columns: column title, breakpoint year, slope, and difference in  slope. Unfortunately, when one of the columns doesn't have a breakpoint the code stops and provides?an error or warning. I would like the loop to keep running regardless of the error/warning but I want it to write that it did encounter an error or warning.  Based on my needs I found that tryCatch appears to do what I need. I have looked at multiple examples, blogs, etc. as well as gone over the documentation and Hadley Wickham's document in Advanced R but I am still a novice when it comes to coding and I am having  a hard time getting the code to work properly. Below is the code I have developed thus far (with a test dataset as an example):

#Creating a test dataset
Year<- c(2000, 2001, 2002, 2003, 2004)
Lake1<- c(2, 4, 5, 2, 1)
Lake2<- c(1, 3, -1, 4, -2)
Lake3<- c(1, 2, 5, -3, 1)
mydata<- data.frame(Year, Lake1, Lake2, Lake3)

#Running a for loop that indicates when an error or warning occurs
y<- mydata[,2:4]
year <- mydata$Year
regimeshift <- data.frame()
for (i in 1:3){
? tryCatch({
? ? y.val <- y[,i]
? ? lin.reg <- lm(y.val~year, mydata)
? ? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
? ? RSyear <- summary(seg.reg)$psi [1,2]
? ? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]?
? ? SlopeDiff <- summary(seg.reg)$coefficients[3,1]?
? ? new.regimeshift <- data.frame(RSyear=RSyear, SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
? ? rownames(new.regimeshift) <- colnames(y)[i]
? ? regimeshift <- rbind(regimeshift,new.regimeshift)
? ? print(regimeshift)
? }, error= function(e) {cat("Error", "\n")},
? ? warning= function(w) {cat("Warning", "\n")})
}

Any ideas or suggestions you might have are greatly?appreciated!

Bailey Hewitt
PhD Candidate
York University
Ontario, Canada
______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide  http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
#
On 5/22/2018 11:32 AM, Bailey Hewitt wrote:
My suggestion is that you get your code to work on a single iteration 
before running in a loop and wrapping the code in tryCatch

I ran your "set-up" code (before the loop).  then extracted your code 
from the loop, set i <- 1, and then ran the code.  Here is what I got:

 > i <- 1
 >     y.val <- y[,i]
 >     lin.reg <- lm(y.val~year, mydata)
 >     seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, 
control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
Warning message:
No breakpoint estimated
 >     RSyear <- summary(seg.reg)$psi [1,2]
 >     SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]
 >     SlopeDiff <- summary(seg.reg)$coefficients[3,1]
Error in summary(seg.reg)$coefficients[3, 1] : subscript out of bounds
 >     new.regimeshift <- data.frame(RSyear=RSyear, 
SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
Error in data.frame(RSyear = RSyear, SlopeRegime1 = SlopeRegime1, 
SlopeDiff = SlopeDiff) :
   object 'SlopeDiff' not found
 >     rownames(new.regimeshift) <- colnames(y)[i]
Error in rownames(new.regimeshift) <- colnames(y)[i] :
   object 'new.regimeshift' not found
 >     regimeshift <- rbind(regimeshift,new.regimeshift)
Error in rbind(regimeshift, new.regimeshift) :
   object 'new.regimeshift' not found
 >     print(regimeshift)

the immediate problems that show up are
1.  there is no variable psi the summary(seg.reg) object (RSyear is NULL).
2.  the coefficients[] matrix in the summary(seg.reg) object does not 
have 3 rows, so you get an error and SlopeDiff is not created.

You need to correct these problems, and any others, so that your code 
runs correctly when there are no data problems.  Then you can worry 
about trapping errors in the case where there are data problems.

Hope this is helpful,

Dan
#
Hi Dan,

Thank you for trying this out! I have been able to use the segmented.lm code outside of the loop effectively. When it gave you the first warning: No breakpoint estimated, that is what I would expect it to do, and since  there is no breakpoint there aren't any variables in seg.reg. What I would like to have happen is that when I get this warning, I want it to be caught by tryCatch and write "warning" in the matrix, and the loop should continue on. Is there a way to manipulate the code for this or something similar?to happen?

Bert, you are right! My apologies, I did mean to reply all!

Thanks!

Bailey




From: Daniel Nordlund <djnordlund at gmail.com>
Sent: May 22, 2018 3:06 PM
To: Bailey Hewitt; Bert Gunter
Cc: r-help at R-project.org
Subject: Re: [R] Using tryCatch in a for loop
?
On 5/22/2018 11:32 AM, Bailey Hewitt wrote:
My suggestion is that you get your code to work on a single iteration 
before running in a loop and wrapping the code in tryCatch

I ran your "set-up" code (before the loop).? then extracted your code 
from the loop, set i <- 1, and then ran the code.? Here is what I got:

?> i <- 1
?>???? y.val <- y[,i]
?>???? lin.reg <- lm(y.val~year, mydata)
?>???? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, 
control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
Warning message:
No breakpoint estimated
?>???? RSyear <- summary(seg.reg)$psi [1,2]
?>???? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]
?>???? SlopeDiff <- summary(seg.reg)$coefficients[3,1]
Error in summary(seg.reg)$coefficients[3, 1] : subscript out of bounds
?>???? new.regimeshift <- data.frame(RSyear=RSyear, 
SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
Error in data.frame(RSyear = RSyear, SlopeRegime1 = SlopeRegime1, 
SlopeDiff = SlopeDiff) :
?? object 'SlopeDiff' not found
?>???? rownames(new.regimeshift) <- colnames(y)[i]
Error in rownames(new.regimeshift) <- colnames(y)[i] :
?? object 'new.regimeshift' not found
?>???? regimeshift <- rbind(regimeshift,new.regimeshift)
Error in rbind(regimeshift, new.regimeshift) :
?? object 'new.regimeshift' not found
?>???? print(regimeshift)

the immediate problems that show up are
1.? there is no variable psi the summary(seg.reg) object (RSyear is NULL).
2.? the coefficients[] matrix in the summary(seg.reg) object does not 
have 3 rows, so you get an error and SlopeDiff is not created.

You need to correct these problems, and any others, so that your code 
runs correctly when there are no data problems.? Then you can worry 
about trapping errors in the case where there are data problems.

Hope this is helpful,

Dan
#
After Dan's suggestions I went back to testing the code outside the for loop again and realized I haven't included a lake that WILL have a breakpoint in my "test" dataset. Below I provide data where some of the lakes produce warnings and others produce results. The output is the matrix I want (as I described previously) but the "warning" is printed outside the final matrix. So I get results for lakes 2 and 4 but there isn't a warning written for Lake 1 and 3 in the final matrix. This is much closer so thank you Dan! Is there any way to get the final result with "warning" written in for lakes without breakpoints so that I can write it into a .csv?


Bailey
#
Data and code as promised:

#Creating a test dataset
Year<- c(2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014)
Lake1<- c(2, 4, 5, 2, 1, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake2<- c(1, 3, -1, 4, -2, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake3<- c(1, 2, 5, -3, 1, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake4<- c(1, 1, 1, 1, 1, 1, 1, 250, 240, 240, 240, 240, 240, 239, 255)
mydata<- data.frame(Year, Lake1, Lake2, Lake3, Lake4)

#Running a for loop that indicates when an error or warning occurs
y<- mydata[,2:5]
year <- mydata$Year
regimeshift <- data.frame()
for (i in 1:4){
  tryCatch({
    y.val <- y[,i]
    lin.reg <- lm(y.val~year, mydata)
    seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
    RSyear <- summary(seg.reg)$psi [1,2]
    SlopeRegime1 <- summary(seg.reg)$coefficients[2,1] 
    SlopeDiff <- summary(seg.reg)$coefficients[3,1] 
    new.regimeshift <- data.frame(RSyear=RSyear, SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
    rownames(new.regimeshift) <- colnames(y)[i]
    regimeshift <- rbind(regimeshift,new.regimeshift)
    print(regimeshift)
  }, error= function(e) {cat("Error", "\n")},
    warning= function(w) {cat("Warning", "\n")})
}

Bailey




From: Daniel Nordlund <djnordlund at gmail.com>
Sent: May 22, 2018 3:06 PM
To: Bailey Hewitt; Bert Gunter
Cc: r-help at R-project.org
Subject: Re: [R] Using tryCatch in a for loop
?
On 5/22/2018 11:32 AM, Bailey Hewitt wrote:
My suggestion is that you get your code to work on a single iteration 
before running in a loop and wrapping the code in tryCatch

I ran your "set-up" code (before the loop).? then extracted your code 
from the loop, set i <- 1, and then ran the code.? Here is what I got:

?> i <- 1
?>???? y.val <- y[,i]
?>???? lin.reg <- lm(y.val~year, mydata)
?>???? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, 
control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
Warning message:
No breakpoint estimated
?>???? RSyear <- summary(seg.reg)$psi [1,2]
?>???? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]
?>???? SlopeDiff <- summary(seg.reg)$coefficients[3,1]
Error in summary(seg.reg)$coefficients[3, 1] : subscript out of bounds
?>???? new.regimeshift <- data.frame(RSyear=RSyear, 
SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
Error in data.frame(RSyear = RSyear, SlopeRegime1 = SlopeRegime1, 
SlopeDiff = SlopeDiff) :
?? object 'SlopeDiff' not found
?>???? rownames(new.regimeshift) <- colnames(y)[i]
Error in rownames(new.regimeshift) <- colnames(y)[i] :
?? object 'new.regimeshift' not found
?>???? regimeshift <- rbind(regimeshift,new.regimeshift)
Error in rbind(regimeshift, new.regimeshift) :
?? object 'new.regimeshift' not found
?>???? print(regimeshift)

the immediate problems that show up are
1.? there is no variable psi the summary(seg.reg) object (RSyear is NULL).
2.? the coefficients[] matrix in the summary(seg.reg) object does not 
have 3 rows, so you get an error and SlopeDiff is not created.

You need to correct these problems, and any others, so that your code 
runs correctly when there are no data problems.? Then you can worry 
about trapping errors in the case where there are data problems.

Hope this is helpful,

Dan
#
No. If your ouput is a numeric "matrix", it cannot include alpha. Columns
in a data frame can be of different classes, but each column must be single
class.

and finally, of course, see ?cat -- I think you are misusing it. If you
simply want to return "somestuff", your function should be:

function(w) {"somestuff"}

not

function(w) {cat("somestuff")}

As usual, apologies if I have misunderstood. Caveat emptor.

Cheers,
Bert



Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, May 22, 2018 at 1:11 PM, Bailey Hewitt <bailster at hotmail.com> wrote:

            

  
  
#
Ah ok?this solves it for me, thank you!?

Bailey




From: Bert Gunter <bgunter.4567 at gmail.com>
Sent: May 22, 2018 4:42 PM
To: Bailey Hewitt
Cc: Daniel Nordlund; r-help at R-project.org
Subject: Re: [R] Using tryCatch in a for loop
?  








No. If your ouput is a numeric "matrix", it cannot include alpha. Columns in a data frame can be of different classes, but each column must be single class.

 and finally, of course, see ?cat -- I think you are misusing it. If you simply want to return "somestuff", your function should be:

 function(w) {"somestuff"}

 not 

 function(w) {cat("somestuff")}

 As usual, apologies if I have misunderstood. Caveat emptor.

 Cheers,
 Bert






Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, May 22, 2018 at 1:11 PM, Bailey Hewitt <bailster at hotmail.com> wrote:
Data and code as promised:

#Creating a test dataset
Year<- c(2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014)
Lake1<- c(2, 4, 5, 2, 1, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake2<- c(1, 3, -1, 4, -2, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake3<- c(1, 2, 5, -3, 1, 1, 2, 3, 4, 5, 6, 2, 3, 1, 2)
Lake4<- c(1, 1, 1, 1, 1, 1, 1, 250, 240, 240, 240, 240, 240, 239, 255)
mydata<- data.frame(Year, Lake1, Lake2, Lake3, Lake4)

#Running a for loop that indicates when an error or warning occurs
y<- mydata[,2:5]
year <- mydata$Year
regimeshift <- data.frame()
for (i in 1:4){
? tryCatch({
? ? y.val <- y[,i]
? ? lin.reg <- lm(y.val~year, mydata)
? ? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
? ? RSyear <- summary(seg.reg)$psi [1,2]
? ? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1] 
? ? SlopeDiff <- summary(seg.reg)$coefficients[3,1] 
? ? new.regimeshift <- data.frame(RSyear=RSyear, SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
? ? rownames(new.regimeshift) <- colnames(y)[i]
? ? regimeshift <- rbind(regimeshift,new.regimeshift)
? ? print(regimeshift)
? }, error= function(e) {cat("Error", "\n")},
? ? warning= function(w) {cat("Warning", "\n")})
}

Bailey




From: Daniel Nordlund <djnordlund at gmail.com>
Sent: May 22, 2018 3:06 PM
To: Bailey Hewitt; Bert Gunter


Cc: r-help at R-project.org
Subject: Re: [R] Using tryCatch in a for loop
?
On 5/22/2018 11:32 AM, Bailey Hewitt wrote:
My suggestion is that you get your code to work on a single iteration 
before running in a loop and wrapping the code in tryCatch

I ran your "set-up" code (before the loop).? then extracted your code 
from the loop, set i <- 1, and then ran the code.? Here is what I got:

?> i <- 1
?>???? y.val <- y[,i]
?>???? lin.reg <- lm(y.val~year, mydata)
?>???? seg.reg <- segmented.lm(lin.reg, seg.Z = ~ year, psi = NA, 
control = seg.control(stop.if.error = FALSE, n.boot = 0, it.max = 20))
Warning message:
No breakpoint estimated
?>???? RSyear <- summary(seg.reg)$psi [1,2]
?>???? SlopeRegime1 <- summary(seg.reg)$coefficients[2,1]
?>???? SlopeDiff <- summary(seg.reg)$coefficients[3,1]
Error in summary(seg.reg)$coefficients[3, 1] : subscript out of bounds
?>???? new.regimeshift <- data.frame(RSyear=RSyear, 
SlopeRegime1=SlopeRegime1, SlopeDiff=SlopeDiff)
Error in data.frame(RSyear = RSyear, SlopeRegime1 = SlopeRegime1, 
SlopeDiff = SlopeDiff) :
?? object 'SlopeDiff' not found
?>???? rownames(new.regimeshift) <- colnames(y)[i]
Error in rownames(new.regimeshift) <- colnames(y)[i] :
?? object 'new.regimeshift' not found
?>???? regimeshift <- rbind(regimeshift,new.regimeshift)
Error in rbind(regimeshift, new.regimeshift) :
?? object 'new.regimeshift' not found
?>???? print(regimeshift)

the immediate problems that show up are
1.? there is no variable psi the summary(seg.reg) object (RSyear is NULL).
2.? the coefficients[] matrix in the summary(seg.reg) object does not 
have 3 rows, so you get an error and SlopeDiff is not created.

You need to correct these problems, and any others, so that your code 
runs correctly when there are no data problems.? Then you can worry 
about trapping errors in the case where there are data problems.

Hope this is helpful,

Dan