Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
Memory leakage/violation?
7 messages · Thomas Lumley, Uwe Ligges, Brian Ripley +1 more
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries). -thomas
On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
Thomas Lumley wrote:
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries).
Cannot reproduce either (using R-2.1.1 and an older version of R-devel, though). Maybe a compiler issue? Henrik, do you use exactly the compiler set up mentioned in the manuals? Which version of gcc? Did your emember to replace the f771.exe? Uwe
-thomas On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
On Sat, 27 Aug 2005, Uwe Ligges wrote:
Thomas Lumley wrote:
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries).
Cannot reproduce either (using R-2.1.1 and an older version of R-devel, though). Maybe a compiler issue? Henrik, do you use exactly the compiler set up mentioned in the manuals? Which version of gcc? Did your emember to replace the f771.exe?
It would be better to use gcc-3.4.4 as recommended in the current manual. However, I read this that Henrik was using the binary from CRAN which I think Duncan M is now building with that. Another possibility is a broken version of a Windows DLL, such as msvcrt.dll.
Uwe
-thomas On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Thank you Thomas and Uwe for this. This is really odd, because today I can neither reproduce it myself (I rebooted my computer this morning). I try to recall what I did yesterday: I did _not_ reboot my machine, but I did install the latest binaries of R 2.1.1 and 2.2.0dev from CRAN. I did close all R sessions and restarted them by R --vanilla (only one at the time) and got the same errors over and over for hours (trust me, I was really frustrated with my analysis). Maybe a reboot would have solved it - I should know better and try that first! A worse scenario is that the hardware was overheated or starts to fall apart. To answer you question Uwe, I don't know about the compiler settings since I got the pre-build binaries from CRAN. Best wishes Henrik
Uwe Ligges wrote:
Thomas Lumley wrote:
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries).
Cannot reproduce either (using R-2.1.1 and an older version of R-devel, though). Maybe a compiler issue? Henrik, do you use exactly the compiler set up mentioned in the manuals? Which version of gcc? Did your emember to replace the f771.exe? Uwe
-thomas On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu.
Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Prof Brian Ripley wrote:
On Sat, 27 Aug 2005, Uwe Ligges wrote:
Thomas Lumley wrote:
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries).
Cannot reproduce either (using R-2.1.1 and an older version of R-devel, though). Maybe a compiler issue? Henrik, do you use exactly the compiler set up mentioned in the manuals? Which version of gcc? Did your emember to replace the f771.exe?
It would be better to use gcc-3.4.4 as recommended in the current manual. However, I read this that Henrik was using the binary from CRAN which I think Duncan M is now building with that. Another possibility is a broken version of a Windows DLL, such as msvcrt.dll.
Thank you for this.
I scanned my computer for possible msvcrt.dll (Microsoft (R) C Runtime
Library) files and found:
C:\Windows\system32\msvcrt.dll
v7.0.2600.1106 (xpsp1.020828-1920), 323 072 bytes
C:\WINDOWS\$NtServicePackUninstall$\msvcrt.dll
v7.0.2600.0 (xpclient.010817-1148), 322 560 bytes
(plus a few versions of the same in "temporary" directories).
As I mentioned in the email I sent recently, I cannot reproduce the
error today after a fresh reboot (and letting the computer rest for 12
hours).
Best
Henrik
Uwe
-thomas On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when
subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I
let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu.
Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
And... hopefully my final admission that I did not do my background research; the system I'm working with and borrowed from a colleague (since my own is on service) is not a Windows XP Pro _SP2_, but apparently _SP1_. It's not worth troubleshooting much until I get an update here. Thank you Brian for making me aware of this by mentioning a possible broken DLL. Thanks you all for taking the time on this one Henrik
Henrik Bengtsson wrote:
Prof Brian Ripley wrote:
On Sat, 27 Aug 2005, Uwe Ligges wrote:
Thomas Lumley wrote:
I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred tries), or running under Valgrind on AMD64 Linux (in four or five tries).
Cannot reproduce either (using R-2.1.1 and an older version of R-devel, though). Maybe a compiler issue? Henrik, do you use exactly the compiler set up mentioned in the manuals? Which version of gcc? Did your emember to replace the f771.exe?
It would be better to use gcc-3.4.4 as recommended in the current manual. However, I read this that Henrik was using the binary from CRAN which I think Duncan M is now building with that. Another possibility is a broken version of a Windows DLL, such as msvcrt.dll.
Thank you for this.
I scanned my computer for possible msvcrt.dll (Microsoft (R) C Runtime
Library) files and found:
C:\Windows\system32\msvcrt.dll
v7.0.2600.1106 (xpsp1.020828-1920), 323 072 bytes
C:\WINDOWS\$NtServicePackUninstall$\msvcrt.dll
v7.0.2600.0 (xpclient.010817-1148), 322 560 bytes
(plus a few versions of the same in "temporary" directories).
As I mentioned in the email I sent recently, I cannot reproduce the
error today after a fresh reboot (and letting the computer rest for 12
hours).
Best
Henrik
Uwe
-thomas On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
Hi,
I've spotted a possible memory leakage/violation in the latest R v2.1.1
patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
I first caught it deep down in a nested svd algorithm when
subtracting a
double 'c' from a integer vector 'a' where both had finite values but
when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
not. Different runs with the identical data would introduce NaNs at
random positions, but not all the time.
Troubleshooting is after a couple of hours still at v0.5, but here is a
script that generates the strange behavior on the above R setups. I
let
the script speak for itself. Note that both the script 'strange.R' and
the data 'strange.RData' is online too, see code below.
People on other systems (but also on Windows), could you please try it
and see if you can reproduce what I get.
Cheers
Henrik
# The following was tested on: Windows XP Pro SP2 Eng with
# i) R Version 2.1.1 Patched (2005-08-25)
# ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
# Start 'R --vanilla' and source() this script, i.e.
# source("http://www.maths.lth.se/help/R/strange.R")
# If you do not get any errors, retry a few times.
foo <- function(x) {
print(list(
name=as.character(substitute(x)),
storage.mode=storage.mode(x),
na=any(is.na(x)),
nan=any(is.nan(x)),
inf=any(is.infinite(x)),
ok=all(is.finite(a))
))
print(length(x))
print(summary(x))
}
# Load data from a complicated "non-reproducible" algorithm.
# The below errors occur also when data is not
# saved and then reloaded from file. Data was generated in
# R v2.1.1 patched (see above).
if (file.exists("strange.RData")) {
load("strange.RData")
} else {
load(url("http://www.maths.lth.se/help/R/strange.RData"))
}
# First glance at data...
foo(a)
foo(c)
## $name
## [1] "a"
##
## $storage.mode
## [1] "integer"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 15000
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 41.0 51.0 63.0 292.2 111.0 65170.0
## $name
## [1] "c"
##
## $storage.mode
## [1] "double"
##
## $na
## [1] FALSE
##
## $nan
## [1] FALSE
##
## $inf
## [1] FALSE
##
## $ok
## [1] TRUE
##
## [1] 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 53.43 53.43 53.43 53.43 53.43 53.43
##
# But, trying the following, will result in
# no-reproducible error messages. Sometimes
# it errors at kk==1, sometimes at kk >> 1.
# Also, look at the different output for
# different kk:s.
for (kk in 1:100) {
cat("kk=",kk, "\n")
print(summary(a-c))
}
## kk= 1
## Min. 1st Qu. Median Mean 3rd Qu.
Max.
## -7.741e+307 -2.431e+00 9.569e+00 5.757e+01
## kk= 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 3
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 57.570 65120.000
## kk= 4
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 5
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -12.430 -2.431 9.569 238.700 57.570 65120.000
## kk= 6
## Error in quantile.default(object) : missing values and NaN's
## not allowed if 'na.rm' is FALSE
## Comments: If you shorten down 'a', the bug occurs less frequently.
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel