Skip to content
Prev 51919 / 63424 Next
Original Raw

Is it possible to increase MAX_NUM_DLLS in future R releases?

Henrik Bengtsson 
Isn't the problem in Qin's example that unloadNamespace("scde") only
unloads 'scde' but none of its package dependencies that were loaded
when 'scde' was loaded.  For example:

$ R --vanilla
$Depends
[1] "R (>= 3.0.0), flexmix"

$Imports
[1] "Rcpp (>= 0.10.4), RcppArmadillo (>= 0.5.400.2.0), mgcv, Rook, rjson, MASS,
 Cairo, RColorBrewer, edgeR, quantreg, methods, nnet, RMTstat, extRemes, pcaMet
hods, BiocParallel, parallel"
[1] "flexmix"       "Rcpp"          "edgeR"         "splines"
 [5] "BiocGenerics"  "MASS"          "BiocParallel"  "scde"
 [9] "lattice"       "rjson"         "brew"          "RcppArmadillo"
[13] "minqa"         "distillery"    "car"           "tools"
[17] "Rook"          "Lmoments"      "nnet"          "parallel"
[21] "pbkrtest"      "RMTstat"       "grid"          "Biobase"
[25] "nlme"          "mgcv"          "quantreg"      "modeltools"
[29] "MatrixModels"  "lme4"          "Matrix"        "nloptr"
[33] "RColorBrewer"  "extRemes"      "limma"         "pcaMethods"
[37] "stats4"        "SparseM"       "Cairo"
[1] "Cairo"         "parallel"      "limma"         "edgeR"
 [5] "MASS"          "rjson"         "Rcpp"          "grid"
 [9] "lattice"       "Matrix"        "SparseM"       "quantreg"
[13] "nnet"          "nlme"          "mgcv"          "Biobase"
[17] "pcaMethods"    "splines"       "minqa"         "nloptr"
[21] "lme4"          "extRemes"      "RcppArmadillo" "tools"
[25] "Rook"          "scde"


If you unload these namespaces, I think the DLLs will also be
detached; or at least they should if packages implement an .onUnload()
with a dyn.unload().  More on this below.


To unloading these added namespaces (with DLLs), they have to be
unloaded in an order that does not break the dependency graph of the
currently loaded packages, because otherwise you'll get errors such
as:
Error in unloadNamespace("quantreg") :
  namespace 'quantreg' is imported by 'car', 'scde' so cannot be unloaded

I don't know if there exist a function that unloads the namespaces in
the proper order, but here is a brute-force version:

unloadNamespaces <- function(ns, ...) {
  while (length(ns) > 0) {
    ns0 <- loadedNamespaces()
    for (name in ns) {
      try(unloadNamespace(name), silent=TRUE)
    }
    ns1 <- loadedNamespaces()
    ## No namespace was unloaded?
    if (identical(ns1, ns0)) break
    ns <- intersect(ns, ns1)
  }
  if (length(ns) > 0) stop("Failed to unload namespace: ",
paste(sQuote(ns), collapse=", "))
} # unloadNamespaces()


When I run the above on R 3.3.0 patched on Windows, I get:
now dyn.unload("C:/Users/hb/R/win-library/3.3/scde/libs/x64/scde.dll") ...
[1] "grDevices" "utils"     "stats"     "datasets"  "base"      "graphics"
[7] "methods"
[1] TRUE


However, there are some namespaces for which the DLLs are still loaded:
[1] "Cairo"         "edgeR"         "extRemes"      "minqa"
 [5] "nloptr"        "pcaMethods"    "quantreg"      "Rcpp"
 [9] "RcppArmadillo" "rjson"         "Rook"          "SparseM"


If we look for .onUnload() in packages that load DLLs, we find that
the following does not have an .onUnload() and therefore probably does
neither call dyn.unload() when the package is unloaded:
+   ns <- getNamespace(pkg)
+   exists(".onUnload", envir=ns, inherits=FALSE)
+ })])
 [1] "Cairo"         "edgeR"         "extRemes"      "minqa"
 [5] "nloptr"        "pcaMethods"    "quantreg"      "Rcpp"
 [9] "RcppArmadillo" "rjson"         "Rook"          "SparseM"


That doesn't look like a coincident to me.  Maybe `R CMD check` should
in addition to checking that the namespace of a package can be
unloaded also assert that it unloads whatever DLL a package loads.
Something like:

* checking whether the namespace can be unloaded cleanly ... WARNING
  Unloading the namespace does not unload DLL

At least I don't think this is tested for, e.g.
https://cran.r-project.org/web/checks/check_results_Cairo.html and
https://cran.r-project.org/web/checks/check_results_Rcpp.html.

/Henrik


On Mon, May 9, 2016 at 11:57 PM, Martin Maechler
<maechler at stat.math.ethz.ch> wrote:

Thread (7 messages)

Qin Zhu Is it possible to increase MAX_NUM_DLLS in future R releases? May 2 Martin Maechler Is it possible to increase MAX_NUM_DLLS in future R releases? May 4 Brian Ripley Is it possible to increase MAX_NUM_DLLS in future R releases? May 4 Martin Morgan Is it possible to increase MAX_NUM_DLLS in future R releases? May 4 Qin Zhu Is it possible to increase MAX_NUM_DLLS in future R releases? May 6 Martin Maechler Is it possible to increase MAX_NUM_DLLS in future R releases? May 9 Henrik Bengtsson Is it possible to increase MAX_NUM_DLLS in future R releases? May 10