Dear r-help,
Somewhere in my innocuous function to rotate an object in Cartesian space
I've created a monster that completely locks up my computer (requires a
hard reset every time). I don't know if this is useful description to
anyone - the mouse still responds, but not the keyboard and not windows
explorer.
The script only does this when the input matrix is large, and so my initial
runs didn't catch it as I used a smaller matrix to speed up the test runs.
When I tried an input matrix with a number of dimensions in the same order
of magnitude as the data I want to read in, R and my computer choked. This
was a surprise for me, as I've always been able to break execution in the
past or do other tasks. So i tried it again, and still no dice.
Now I need the function to work as subsequent functions/functionality are
dependent, and I can't see anything on the face of it that would likely
cause the issue.
Any insight on why this happens in general or specifically in my case are
appreciated. Running R 15.2, Platform: x86_64-w64-mingw32/x64 (64-bit) on a
windows 7 machine with 4 mb RAM. In the meantime I suppose I'll write a
loop to do this function piece-wise for larger data and see if that helps.
Script is attached and appended below.
Thanks
Ben Caldwell
#compass to polar coordinates
compass2polar <- function(x) {-x+90}
#degrees (polar) to radians
Deg2Rad <- function(x) {(x*pi)/180}
# radians to degrees
Rad2Deg <- function (rad) (rad/pi)*180
# polar to cartesian coordinates - assumes degrees those from a compass.
output is a list, x & y of equal length
Pol2Car <- function(distance,deg) {
rad <- Deg2Rad(compass2polar(deg))
rad <- rep(rad, length(distance))
x <- ifelse(is.na(distance), NA, distance * cos(rad))
y <- ifelse(is.na(distance), NA, distance * sin(rad))
x<-round(x,2)
y<-round(y,2)
cartes<- list(x,y)
name<-c('x','y')
names(cartes)<-name
cartes
}
#rotate an object, with assumed origin at 0,0, in any number of degrees
rotate <- function(x,y,tilt){ 8
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
n <- length(or.deg)
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
# browser()
tilt.in <- tilt + or.deg
xy<-Pol2Car (distance=rotate.dis, deg=tilt.in)
# if(abs(tilt) >= 0) {
# shift.frame <- cbind(xy$x, xy$y)
# shift.frame.val <- shift.frame[shift.frame[,2]==min(shift.frame[,2]),]
# shift.x<- shift.frame.val[1] * -1
# shift.y<- shift.frame.val[2] * -1
# x<-xy$x + shift.x
# y<-xy$y + shift.y
# }
# name <- c('x', 'y')
# xy<-list(x,y)
# names(xy)<-name
xy
}
x <- seq(0,5, .5)
y <- seq(0,5, .5)
z <- seq(0,5, .5)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
plot(dquad$x, dquad$y, xlim=c(-25,25), ylim=c(-25,25))
#this works fine
rotated<-rotate(dquad$x, dquad$y, 45)
points(rotated$x, rotated$y, col='green')
# profiling of both time and memory
Rprof(?myFunction.out?, memory.profiling=T)
y <- myFunction(x)
Rprof(NULL)
summaryRprof(?myFunction.out?, memory=?both?)
#############
x <- seq(0,5, .1)
y <- seq(0,5, .1)
z <- seq(0,5, .1)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
# running the below locks up my machine (commented out to avoid accidental
run)
# rotated<-rotate(dquad$x, dquad$y, 45)
Function completely locks up my computer if the input is too big
5 messages · Peter Alspach, William Dunlap, Benjamin Caldwell +1 more
Tena koe Benjamin
I haven't looked at you code in detail, but in general ifelse is slow and can generally be avoided. For example,
ben <- 1:10^7
system.time(BEN <- ifelse(ben<10, NA, -ben))
user system elapsed
1.31 0.24 1.56
system.time({BEN1 <- -ben; BEN1[BEN1> -10] <- NA})
user system elapsed
0.17 0.03 0.20
all.equal(BEN, BEN1)
[1] TRUE
HTH ...
Peter Alspach
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Benjamin Caldwell
Sent: Wednesday, 6 March 2013 10:18 a.m.
To: r-help
Subject: [R] Function completely locks up my computer if the input is too big
Dear r-help,
Somewhere in my innocuous function to rotate an object in Cartesian space I've created a monster that completely locks up my computer (requires a hard reset every time). I don't know if this is useful description to anyone - the mouse still responds, but not the keyboard and not windows explorer.
The script only does this when the input matrix is large, and so my initial runs didn't catch it as I used a smaller matrix to speed up the test runs.
When I tried an input matrix with a number of dimensions in the same order of magnitude as the data I want to read in, R and my computer choked. This was a surprise for me, as I've always been able to break execution in the past or do other tasks. So i tried it again, and still no dice.
Now I need the function to work as subsequent functions/functionality are dependent, and I can't see anything on the face of it that would likely cause the issue.
Any insight on why this happens in general or specifically in my case are appreciated. Running R 15.2, Platform: x86_64-w64-mingw32/x64 (64-bit) on a windows 7 machine with 4 mb RAM. In the meantime I suppose I'll write a loop to do this function piece-wise for larger data and see if that helps.
Script is attached and appended below.
Thanks
Ben Caldwell
#compass to polar coordinates
compass2polar <- function(x) {-x+90}
#degrees (polar) to radians
Deg2Rad <- function(x) {(x*pi)/180}
# radians to degrees
Rad2Deg <- function (rad) (rad/pi)*180
# polar to cartesian coordinates - assumes degrees those from a compass.
output is a list, x & y of equal length
Pol2Car <- function(distance,deg) {
rad <- Deg2Rad(compass2polar(deg))
rad <- rep(rad, length(distance))
x <- ifelse(is.na(distance), NA, distance * cos(rad))
y <- ifelse(is.na(distance), NA, distance * sin(rad))
x<-round(x,2)
y<-round(y,2)
cartes<- list(x,y)
name<-c('x','y')
names(cartes)<-name
cartes
}
#rotate an object, with assumed origin at 0,0, in any number of degrees
rotate <- function(x,y,tilt){ 8
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
n <- length(or.deg)
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
# browser()
tilt.in <- tilt + or.deg
xy<-Pol2Car (distance=rotate.dis, deg=tilt.in)
# if(abs(tilt) >= 0) {
# shift.frame <- cbind(xy$x, xy$y)
# shift.frame.val <- shift.frame[shift.frame[,2]==min(shift.frame[,2]),]
# shift.x<- shift.frame.val[1] * -1
# shift.y<- shift.frame.val[2] * -1
# x<-xy$x + shift.x
# y<-xy$y + shift.y
# }
# name <- c('x', 'y')
# xy<-list(x,y)
# names(xy)<-name
xy
}
x <- seq(0,5, .5)
y <- seq(0,5, .5)
z <- seq(0,5, .5)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
plot(dquad$x, dquad$y, xlim=c(-25,25), ylim=c(-25,25))
#this works fine
rotated<-rotate(dquad$x, dquad$y, 45)
points(rotated$x, rotated$y, col='green')
# profiling of both time and memory
Rprof("myFunction.out", memory.profiling=T)
y <- myFunction(x)
Rprof(NULL)
summaryRprof("myFunction.out", memory="both")
#############
x <- seq(0,5, .1)
y <- seq(0,5, .1)
z <- seq(0,5, .1)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
# running the below locks up my machine (commented out to avoid accidental
run)
# rotated<-rotate(dquad$x, dquad$y, 45)
The contents of this e-mail are confidential and may be ...{{dropped:14}}
I see you have profiling calls in there. Have you used them?
It is often fruitful to see how the time for a function grows as size of the input or output grows. Have you tried that?
A concrete suggestion is to change
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
to
or.deg[is.na(or.deg)] <- 0
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf
Of Benjamin Caldwell
Sent: Tuesday, March 05, 2013 1:18 PM
To: r-help
Subject: [R] Function completely locks up my computer if the input is too big
Dear r-help,
Somewhere in my innocuous function to rotate an object in Cartesian space
I've created a monster that completely locks up my computer (requires a
hard reset every time). I don't know if this is useful description to
anyone - the mouse still responds, but not the keyboard and not windows
explorer.
The script only does this when the input matrix is large, and so my initial
runs didn't catch it as I used a smaller matrix to speed up the test runs.
When I tried an input matrix with a number of dimensions in the same order
of magnitude as the data I want to read in, R and my computer choked. This
was a surprise for me, as I've always been able to break execution in the
past or do other tasks. So i tried it again, and still no dice.
Now I need the function to work as subsequent functions/functionality are
dependent, and I can't see anything on the face of it that would likely
cause the issue.
Any insight on why this happens in general or specifically in my case are
appreciated. Running R 15.2, Platform: x86_64-w64-mingw32/x64 (64-bit) on a
windows 7 machine with 4 mb RAM. In the meantime I suppose I'll write a
loop to do this function piece-wise for larger data and see if that helps.
Script is attached and appended below.
Thanks
Ben Caldwell
#compass to polar coordinates
compass2polar <- function(x) {-x+90}
#degrees (polar) to radians
Deg2Rad <- function(x) {(x*pi)/180}
# radians to degrees
Rad2Deg <- function (rad) (rad/pi)*180
# polar to cartesian coordinates - assumes degrees those from a compass.
output is a list, x & y of equal length
Pol2Car <- function(distance,deg) {
rad <- Deg2Rad(compass2polar(deg))
rad <- rep(rad, length(distance))
x <- ifelse(is.na(distance), NA, distance * cos(rad))
y <- ifelse(is.na(distance), NA, distance * sin(rad))
x<-round(x,2)
y<-round(y,2)
cartes<- list(x,y)
name<-c('x','y')
names(cartes)<-name
cartes
}
#rotate an object, with assumed origin at 0,0, in any number of degrees
rotate <- function(x,y,tilt){ 8
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
n <- length(or.deg)
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
# browser()
tilt.in <- tilt + or.deg
xy<-Pol2Car (distance=rotate.dis, deg=tilt.in)
# if(abs(tilt) >= 0) {
# shift.frame <- cbind(xy$x, xy$y)
# shift.frame.val <- shift.frame[shift.frame[,2]==min(shift.frame[,2]),]
# shift.x<- shift.frame.val[1] * -1
# shift.y<- shift.frame.val[2] * -1
# x<-xy$x + shift.x
# y<-xy$y + shift.y
# }
# name <- c('x', 'y')
# xy<-list(x,y)
# names(xy)<-name
xy
}
x <- seq(0,5, .5)
y <- seq(0,5, .5)
z <- seq(0,5, .5)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
plot(dquad$x, dquad$y, xlim=c(-25,25), ylim=c(-25,25))
#this works fine
rotated<-rotate(dquad$x, dquad$y, 45)
points(rotated$x, rotated$y, col='green')
# profiling of both time and memory
Rprof("myFunction.out", memory.profiling=T)
y <- myFunction(x)
Rprof(NULL)
summaryRprof("myFunction.out", memory="both")
#############
x <- seq(0,5, .1)
y <- seq(0,5, .1)
z <- seq(0,5, .1)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
# running the below locks up my machine (commented out to avoid accidental
run)
# rotated<-rotate(dquad$x, dquad$y, 45)
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20130305/08266da3/attachment.pl>
Le mardi 05 mars 2013 ? 15:19 -0800, Benjamin Caldwell a ?crit :
Hi all, Thanks for the suggestions. Updating the function as below to break the problem into chunks seemed to do the trick - perhaps there is a relatively small limit to the size of a vector that R can work with?
On the contrary, that's because the limit supported by R is too high for your computer's memory that the OS tries to allocate too much memory and swaps to death. Admittedly, an OS should be smart enough not to completely freeze in the process, but that's just how it is... (If R did not support such a long vector, you would just get a "nice" error message, that's all.) Regards
Best
rotate <- function(x,y,tilt,threshold){
df.main<-data.frame(x,y)
if(length(x)> threshold){
l <- round(length(x)/ threshold, 0)
dfchunk <- split(df.main, factor(sort(rank(row.names(df.main))%%l)))
n<-length(summary(dfchunk)[,1])
xy<-vector("list", n)
for (i in 1:n){
wk.df <- dfchunk[[i]]
x <- wk.df$x
y <- wk.df$y
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
or.deg[is.na(or.deg)] <- 0
tilt.in <- tilt + or.deg
xy[[i]]<-data.frame(Pol2Car(distance=rotate.dis, deg=tilt.in))
}
xy<-do.call("rbind", xy[1:n])
} else {
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
n <- length(or.deg)
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
tilt.in <- tilt + or.deg
xy<-data.frame(Pol2Car (distance=rotate.dis, deg=tilt.in))
}
xy
}
*Ben Caldwell*
Graduate Fellow
University of California, Berkeley
130 Mulford Hall #3114
Berkeley, CA 94720
Office 223 Mulford Hall
(510)859-3358
On Tue, Mar 5, 2013 at 1:44 PM, Peter Alspach <
Peter.Alspach at plantandfood.co.nz> wrote:
Tena koe Benjamin
I haven't looked at you code in detail, but in general ifelse is slow and
can generally be avoided. For example,
ben <- 1:10^7
system.time(BEN <- ifelse(ben<10, NA, -ben))
user system elapsed
1.31 0.24 1.56
system.time({BEN1 <- -ben; BEN1[BEN1> -10] <- NA})
user system elapsed
0.17 0.03 0.20
all.equal(BEN, BEN1)
[1] TRUE
HTH ...
Peter Alspach
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Benjamin Caldwell
Sent: Wednesday, 6 March 2013 10:18 a.m.
To: r-help
Subject: [R] Function completely locks up my computer if the input is too
big
Dear r-help,
Somewhere in my innocuous function to rotate an object in Cartesian space
I've created a monster that completely locks up my computer (requires a
hard reset every time). I don't know if this is useful description to
anyone - the mouse still responds, but not the keyboard and not windows
explorer.
The script only does this when the input matrix is large, and so my
initial runs didn't catch it as I used a smaller matrix to speed up the
test runs.
When I tried an input matrix with a number of dimensions in the same order
of magnitude as the data I want to read in, R and my computer choked. This
was a surprise for me, as I've always been able to break execution in the
past or do other tasks. So i tried it again, and still no dice.
Now I need the function to work as subsequent functions/functionality are
dependent, and I can't see anything on the face of it that would likely
cause the issue.
Any insight on why this happens in general or specifically in my case are
appreciated. Running R 15.2, Platform: x86_64-w64-mingw32/x64 (64-bit) on a
windows 7 machine with 4 mb RAM. In the meantime I suppose I'll write a
loop to do this function piece-wise for larger data and see if that helps.
Script is attached and appended below.
Thanks
Ben Caldwell
#compass to polar coordinates
compass2polar <- function(x) {-x+90}
#degrees (polar) to radians
Deg2Rad <- function(x) {(x*pi)/180}
# radians to degrees
Rad2Deg <- function (rad) (rad/pi)*180
# polar to cartesian coordinates - assumes degrees those from a compass.
output is a list, x & y of equal length
Pol2Car <- function(distance,deg) {
rad <- Deg2Rad(compass2polar(deg))
rad <- rep(rad, length(distance))
x <- ifelse(is.na(distance), NA, distance * cos(rad))
y <- ifelse(is.na(distance), NA, distance * sin(rad))
x<-round(x,2)
y<-round(y,2)
cartes<- list(x,y)
name<-c('x','y')
names(cartes)<-name
cartes
}
#rotate an object, with assumed origin at 0,0, in any number of degrees
rotate <- function(x,y,tilt){ 8
d2 <- x^2+y^2
rotate.dis<-sqrt(d2)
or.rad <- atan(x/y)
or.deg <- Rad2Deg(or.rad)
n <- length(or.deg)
for(i in 1:n){
if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
}
# browser()
tilt.in <- tilt + or.deg
xy<-Pol2Car (distance=rotate.dis, deg=tilt.in)
# if(abs(tilt) >= 0) {
# shift.frame <- cbind(xy$x, xy$y)
# shift.frame.val <- shift.frame[shift.frame[,2]==min(shift.frame[,2]),]
# shift.x<- shift.frame.val[1] * -1
# shift.y<- shift.frame.val[2] * -1
# x<-xy$x + shift.x
# y<-xy$y + shift.y
# }
# name <- c('x', 'y')
# xy<-list(x,y)
# names(xy)<-name
xy
}
x <- seq(0,5, .5)
y <- seq(0,5, .5)
z <- seq(0,5, .5)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
plot(dquad$x, dquad$y, xlim=c(-25,25), ylim=c(-25,25))
#this works fine
rotated<-rotate(dquad$x, dquad$y, 45)
points(rotated$x, rotated$y, col='green')
# profiling of both time and memory
Rprof("myFunction.out", memory.profiling=T)
y <- myFunction(x)
Rprof(NULL)
summaryRprof("myFunction.out", memory="both")
#############
x <- seq(0,5, .1)
y <- seq(0,5, .1)
z <- seq(0,5, .1)
dquad<-expand.grid(x,y,z)
name<-c("y","x","z")
names(dquad)<-name
# running the below locks up my machine (commented out to avoid accidental
run)
# rotated<-rotate(dquad$x, dquad$y, 45)
The contents of this e-mail are confidential and may be subject to legal
privilege.
If you are not the intended recipient you must not use, disseminate,
distribute or
reproduce all or any part of this e-mail or attachments. If you have
received this
e-mail in error, please notify the sender and delete all material
pertaining to this
e-mail. Any opinion or views expressed in this e-mail are those of the
individual
sender and may not represent those of The New Zealand Institute for Plant
and
Food Research Limited.
[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.