R CMD check: Error in function (env) : could not find function "finalize"

Hi,

does someone else get this error message:

Error in function (env)  : could not find function "finalize"?

I get an error when running examples in R CMD check (v2.6.0; session
info below):
### * invertMap

flush(stderr()); flush(stdout())
### Name: invertMap
### Title: Inverts a read or a write map
### Aliases: invertMap
### Keywords: file IO internal

### ** Examples

# .... [TRUNCATED]
nbrOfCells <- 2600000
readMap <- sample(nbrOfCells)
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"

The error occurs in R CMD check but also when start a fresh R session
and run, in this case, affxparser.Rcheck/affxparser-Ex.R.  It always
occur on the same line.

When I debug(sample), all values passed to the internal function looks
fine.  I've replaced it with runif() and vector("double", nbrOfCells)
and manged to get the error there too.  So, it seems like there are
some memory issues.  I've called gc() just before without success.
There are a lot of other example code processed before this one and
many of them rely on native code, so it could be a memory leak or
something.  However, before troubleshooting more, I wanted to hear if
someone else got the same problem.  Searching google, gives me one hit
in a recent Bioconductor build/check.  From Google cache
http://tinyurl.com/25hkld
## Perform svdImpute using the 3 largest components
result <- pca(metaboliteData, method="svdImpute", nPcs=3, center = TRUE)
Error in function (env)  : could not find function "finalize"
...

Now, the error is not there.

Any ideas?

/Henrik
sessionInfo()
R version 2.6.0 Under development (unstable) (2007-08-23 r42614)
i386-pc-mingw32

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MON
ETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] affxparser_1.9.5

loaded via a namespace (and not attached):
[1] AffymetrixDataTestFiles_0.1.0
Hi Henrik,

"Henrik Bengtsson" <hb at stat.berkeley.edu> writes:
Hi,

does someone else get this error message:

Error in function (env)  : could not find function "finalize"?

I get an error when running examples in R CMD check (v2.6.0; session
info below):
[snip]
The error occurs in R CMD check but also when start a fresh R session
and run, in this case, affxparser.Rcheck/affxparser-Ex.R.  It always
occur on the same line.
So does options(error=recover) help in determining where the error is
coming from?

If you can narrow it down, gctorture may help or running the examples
under valgrind.

+ seth
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
BioC: http://bioconductor.org/
Blog: http://userprimary.net/user/
Hi, thanks Seth and others (I've got some offline replies); all
feedback has been useful indeed.

The short story is that as the author of R.oo I am actually the bad
guy here (but not for long since I'm soon committing a fix for R.oo).

REPRODUCIBLE EXAMPLE #1:
% R --vanilla
library(R.oo)
R.oo v1.2.8 (2006-06-09) successfully loaded. See ?R.oo for help.
detach("package:R.oo")
gc()
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 142543  3.9     350000  9.4   350000  9.4
Vcells  82660  0.7     786432  6.0   478255  3.7

REPRODUCIBLE EXAMPLE #2:
Here is another example without R.oo illustrating what is going on.
% R --vanilla
e <- new.env()
e$foo <- "foo"
e$foo <- 1
e <- new.env()
e$foo <- 1
reg.finalizer(e, function(env) { print(ls.str(envir=env)) })
detach("package:utils")
rm(e)
gc()
Error in print(ls.str(envir = env)) : could not find function "ls.str"
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 158213  4.3     350000  9.4   350000  9.4
Vcells  86800  0.7     786432  6.0   478255  3.7

WHY ONLY WHEN RUNNING R CMD CHECK?
So, the problem I had was with 'affxparser' examples failing in R CMD
check, but not when I tested them manually.  Same thing was happening
with the 'pcaMethods' package.  The common denominator was that both
'affxparser' and 'pcaMethods' had R.oo dependent package in
DESCRIPTION/Suggests; 'affxparser' used Suggests: R.utils (which
depends on R.oo), and 'pcaMethods' used Suggests: aroma.light (which
in turn *suggests* R.utils).  To the best of my understanding, when R
CMD check runs examples, it will load *all* suggested packages, and
when done, detach them.  When the garbage collector later runs and
cleans out objects, the generic function finalize() in R.oo called by
the registered finalize hook is not around anymore.  FYI, if you move
the R.oo-dependent package from Suggests: to Depends:, there is no
longer a problem because then the package is never detached.  It all
makes sense.

CONCLUSION:
When registering finalizers for object using reg.finalizer() there is
always the risk of the finalizer code to be broken because a dependent
package has been detached.

SOLUTION:
At least make the finalizer hook function robust against these things.
 For instance, check if required packages are loaded etc, or just add
a tryCatch() statement.  However, since finalizers are typically used
to deallocate resources, much more effort has to be taken to make sure
that is still work, which is not easy.  For instance, one could make
sure to require() the necessary packages in the finalizer, but that
has side effects and it is not necessarily sufficient, e.g. you might
only load a generic function, but the method for a specific subclass
might be in a package out of your control.  Same problem goes with
explicit namespace calls to generic functions, e.g. R.oo::finalize().
If you have more clever suggestions, please let me know.

SOME MORE DETAILS ON R.OO:
This is what R.oo looks like now:

Object <- function (core = NA) {
  this <- core
  attr(this, ".env") <- new.env()
  class(this) <- "Object"
  attr(this, "...instanciationTime") <- Sys.time()
  reg.finalizer(attr(this, ".env"), function(env) finalize(this))
  this
}

finalize.Object <- function(this, ...) {}

finalize <- function(...) UseMethod("finalize")

As you see, when detaching R.oo, finalize() is no longer around.

Lesson learned!

Cheers

Henrik
Hi Henrik,

"Henrik Bengtsson" <hb at stat.berkeley.edu> writes:

Hi,

does someone else get this error message:

Error in function (env)  : could not find function "finalize"?

I get an error when running examples in R CMD check (v2.6.0; session
info below):
[snip]
The error occurs in R CMD check but also when start a fresh R session
and run, in this case, affxparser.Rcheck/affxparser-Ex.R.  It always
occur on the same line.
So does options(error=recover) help in determining where the error is
coming from?

If you can narrow it down, gctorture may help or running the examples
under valgrind.

+ seth

--
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
BioC: http://bioconductor.org/
Blog: http://userprimary.net/user/

To the best of my understanding, when R
CMD check runs examples, it will load *all* suggested packages, and
when done, detach them.  When the garbage collector later runs and
Not so.  R CMD check just runs the examples in a normal session after 
loading the package being tested.  Examples may themselves attach/load 
suggested packages (and if they are suggested it is likely that either 
examples or vignettes will do so).  After each group of examples 
(\examples from a single help file) any packages which have been attached 
in the course of that group are detached.

Looking at pcaMethods, function robustSvd require()s aroma.light, so this 
will happen in example(robustSvd).

I think a package that sets finalizers probably ought to ensure that they 
are run in its .Last.lib or similar hook.  There is no guarantee that they 
will be detached in a particular order, though.  (R CMD check does detach 
them in an ordering determined from the search path, but users may do 
something different.)  If namespaces are involved, similar considerations 
apply to unloading namespaces (although there are some guarantees on order 
since you cannot unload a namespace which is imported from).  Beyond that, 
you may be able to ensure by setting the environment for the 
finalizer function that what it needs will still be present at 
finalization.

Hi, thanks Seth and others (I've got some offline replies); all
feedback has been useful indeed.

The short story is that as the author of R.oo I am actually the bad
guy here (but not for long since I'm soon committing a fix for R.oo).

REPRODUCIBLE EXAMPLE #1:
% R --vanilla
library(R.oo)
R.oo v1.2.8 (2006-06-09) successfully loaded. See ?R.oo for help.
detach("package:R.oo")
gc()
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
Error in function (env)  : could not find function "finalize"
        used (Mb) gc trigger (Mb) max used (Mb)
Ncells 142543  3.9     350000  9.4   350000  9.4
Vcells  82660  0.7     786432  6.0   478255  3.7

REPRODUCIBLE EXAMPLE #2:
Here is another example without R.oo illustrating what is going on.
% R --vanilla
e <- new.env()
e$foo <- "foo"
e$foo <- 1
e <- new.env()
e$foo <- 1
reg.finalizer(e, function(env) { print(ls.str(envir=env)) })
detach("package:utils")
rm(e)
gc()
Error in print(ls.str(envir = env)) : could not find function "ls.str"
        used (Mb) gc trigger (Mb) max used (Mb)
Ncells 158213  4.3     350000  9.4   350000  9.4
Vcells  86800  0.7     786432  6.0   478255  3.7

WHY ONLY WHEN RUNNING R CMD CHECK?
So, the problem I had was with 'affxparser' examples failing in R CMD
check, but not when I tested them manually.  Same thing was happening
with the 'pcaMethods' package.  The common denominator was that both
'affxparser' and 'pcaMethods' had R.oo dependent package in
DESCRIPTION/Suggests; 'affxparser' used Suggests: R.utils (which
depends on R.oo), and 'pcaMethods' used Suggests: aroma.light (which
in turn *suggests* R.utils).  To the best of my understanding, when R
CMD check runs examples, it will load *all* suggested packages, and
when done, detach them.  When the garbage collector later runs and
cleans out objects, the generic function finalize() in R.oo called by
the registered finalize hook is not around anymore.  FYI, if you move
the R.oo-dependent package from Suggests: to Depends:, there is no
longer a problem because then the package is never detached.  It all
makes sense.

CONCLUSION:
When registering finalizers for object using reg.finalizer() there is
always the risk of the finalizer code to be broken because a dependent
package has been detached.

SOLUTION:
At least make the finalizer hook function robust against these things.
For instance, check if required packages are loaded etc, or just add
a tryCatch() statement.  However, since finalizers are typically used
to deallocate resources, much more effort has to be taken to make sure
that is still work, which is not easy.  For instance, one could make
sure to require() the necessary packages in the finalizer, but that
has side effects and it is not necessarily sufficient, e.g. you might
only load a generic function, but the method for a specific subclass
might be in a package out of your control.  Same problem goes with
explicit namespace calls to generic functions, e.g. R.oo::finalize().
If you have more clever suggestions, please let me know.

SOME MORE DETAILS ON R.OO:
This is what R.oo looks like now:

Object <- function (core = NA) {
 this <- core
 attr(this, ".env") <- new.env()
 class(this) <- "Object"
 attr(this, "...instanciationTime") <- Sys.time()
 reg.finalizer(attr(this, ".env"), function(env) finalize(this))
 this
}

finalize.Object <- function(this, ...) {}

finalize <- function(...) UseMethod("finalize")

As you see, when detaching R.oo, finalize() is no longer around.

Lesson learned!

Cheers

Henrik

On 8/28/07, Seth Falcon <sfalcon at fhcrc.org> wrote:
Hi Henrik,

"Henrik Bengtsson" <hb at stat.berkeley.edu> writes:

Hi,

does someone else get this error message:

Error in function (env)  : could not find function "finalize"?

I get an error when running examples in R CMD check (v2.6.0; session
info below):
[snip]
The error occurs in R CMD check but also when start a fresh R session
and run, in this case, affxparser.Rcheck/affxparser-Ex.R.  It always
occur on the same line.
So does options(error=recover) help in determining where the error is
coming from?

If you can narrow it down, gctorture may help or running the examples
under valgrind.

+ seth

--
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
BioC: http://bioconductor.org/
Blog: http://userprimary.net/user/

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595