Skip to content

Suggested dependencies in context of R CMD check

8 messages · Jan Gorecki, Hadley Wickham, Gabriel Becker +3 more

Original
Jan Gorecki
# 
Dear R team,

Are suggested dependencies mandatory in context of `R CMD check` when
using env var `_R_CHECK_FORCE_SUGGESTS_=FALSE`?

Suggested dependencies are nice because are optional.
But that feature often isn't valid when trying to run `R CMD check` on them.
I would like to use `export _R_CHECK_FORCE_SUGGESTS_=FALSE`, which
according to the comment in `tools/R/check.R`:
Checking can be attempted without them by setting the environment
variable _R_CHECK_FORCE_SUGGESTS_ to a false value.

should be sufficient to run *check* process without installing suggested deps.
If it is not true, then how to differentiate the suggested packages
which are optional from those mandatory?
It would be helpful to have kind of `vignetteBuilder` DESCRIPTION
field called `testChecker`, so DESCRIPTION file could precisely manage
packages dependencies.
That way batch checking packages would be easier, as required dep
metadata would be at hand in description file. Batch checking pkgs
with all their suggests will simply result into testing whole CRAN.

In a single package it can be handled with `if (requireNamespace(.))
test_package(.)` for `testthat` and `knitr`, also with mocking up
`.Rout` files.

But I'm interested into canonical design of a suggested package and
`_R_CHECK_FORCE_SUGGESTS_` env var.
Is there any R core dev team recommendation/suggestion on that? and
don't you thing new field `testChecker` in DESCRIPTION could help for
batch checking pkgs? Installing all suggested packages of all reverse
dependencies doesn't scale.

Jan Gorecki
2 days later
Hadley Wickham
#
On Sat, Apr 2, 2016 at 5:33 AM, Jan G?recki <J.Gorecki at wit.edu.pl> wrote:
In principle, I believe a package should pass R CMD check if no
suggested packages are installed. However, since this is not currently
automatically checked, many packages will fail to cleanly pass R CMD
check if suggested packages are missing. In my experience, it's much
easier to simply install all dependencies of your reverse dependencies
(although this is obviously much easier when you're using a platform
with binary packages available from CRAN). I routinely do this for
hundreds to thousands of packages.

Hadley
Gabriel Becker
# 
Jan and Hadley,

There's also the issue of tests, vignettes, or examples requiring Suggested
packages (one of the core applications of Suggests, in fact). These are all
checked by R CMD check, so to ensure any package which should pass check
would do so without suggested packages installed would require turning all
of those checks off, which takes quite a few of the teeth out of R CMD
check (and thus the assurances granted by knowing a package passes it).
AFAIK (without going off and checking right now) you /could/ force R CMD
check to do this now with a combination of options, though, if that's
really what you want (which would allow your package to pass when
_R_CHECK_FORCE_SUGGESTS_ is set to false).

~G
On Mon, Apr 4, 2016 at 7:25 AM, Hadley Wickham <h.wickham at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Jan Gorecki
    

    

      
      #
    

  


  

      
Hi Hadley and Gabriel,

What do you think about new field in DESCRIPTION?
For me it does perfect sense to have new field that lists those from
Suggested deps which are truly needed to run check. The same way as
vignetteBuilder. So couldn't be escaped with `if (requireNamespace(.))
...`, or it wouldn't make sense to test the package without them.
This field would not be needed if package author would use `if
(requireNamespace(.)) ...` for all Suggested deps used in tests.
I understand this transition would take quite a long time.

Imagine having 20 suggested deps, and only one of them used in tests.
Many of the other suggested can be tied to system level dependencies,
like databases, etc. And when you are testing rev deps you don't have
much control/knowledge over that.
Current process doesn't scale and I don't see any better way to address it.

Jan

        
On 4 April 2016 at 16:36, Gabriel Becker <gmbecker at ucdavis.edu> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 4 April 2016 at 07:25, Hadley Wickham wrote:

        
| On Sat, Apr 2, 2016 at 5:33 AM, Jan G?recki <J.Gorecki at wit.edu.pl> wrote:

        
| 
| In principle, I believe a package should pass R CMD check if no
| suggested packages are installed. However, since this is not currently

The relevant manual says

     The 'Suggests' field uses the same syntax as 'Depends' and lists
  packages that are not necessarily needed.  This includes packages used
  only in examples, tests or vignettes (*note Writing package
  vignettes::), and packages loaded in the body of functions.  E.g.,
  suppose an example(1) from package *foo* uses a dataset from package
  *bar*.  Then it is not necessary to have *bar* use *foo* unless one
  wants to execute all the examples/tests/vignettes: it is useful to have
  *bar*, but not necessary.  Version requirements can be specified, and
  will be used by 'R CMD check'.

and later

   * All packages that are needed(2) to successfully run 'R CMD check'
     on the package must be listed in one of 'Depends' or 'Suggests' or
     'Imports'.  Packages used to run examples or tests conditionally
     (e.g. _via_ 'if(require(PKGNAME))') should be listed in 'Suggests'
     or 'Enhances'.  (This allows checkers to ensure that all the
     packages needed for a complete check are installed.)

| automatically checked, many packages will fail to cleanly pass R CMD
| check if suggested packages are missing.

I consider that to be a bug in those 'many packages'.  It essentially takes
away the usefulness of having a Suggests: to provide a more fine-grained
dependency graph.

So I am with Jan here.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On 04/04/2016 1:35 PM, Dirk Eddelbuettel wrote:

            

      
      
      
    

        
I think I agree with Jan, but not for the reason you state.

Suggests is useful even if "R CMD check" treats it as Depends, because 
most users never need to run "R CMD check".  It allows them to use a 
subset of the functionality of a package without installing tons of 
dependencies.

I agree that packages that fail on examples when Suggested packages are 
missing are broken.  (Using if (require()) to skip particular examples 
isn't failing.)  It would be useful to be able to detect failure; I 
don't think that's easy now with "R CMD check".  That's why you should 
be able to run it with Suggested packages missing.

The ideal situation would be to be able to run all possible combinations 
of missing Suggested packages, but that's probably far too slow to be a 
default.

BTW, I'm not completely sure it needs to be possible to run vignettes 
without the Suggested packages they need.  Vignettes are allowed to 
depend on things that aren't available to all users, and adding all the 
require() tests could make them less clear.

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On 04/04/2016 7:12 PM, Paul Gilbert wrote:

            

      
      
      
    

        
Supposing all dependencies are available, there are several ways to 
decide on pass vs fail.

  - You fail if your examples generate an error.
  - If you have saved example output, you fail if it doesn't match the 
old output.
  - You fail if your tests generate an error, or have different output 
than the optional saved output.

So what if some Suggested files are not available during testing?

The "fail on error" rule would be identical if the Suggested packages 
were missing.  Output of tests and examples will likely change, so if it 
were possible to test without Suggested packages, you'd need a way to 
indicate what changes are allowed.  (But most packages don't save test 
results and don't save example output anyway.)

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Paul Gilbert
    

    

      
      #
    

  


  

      
On 04/04/2016 01:56 PM, Duncan Murdoch wrote:

            

      
      
      
    

        
Perhaps I'm confused, it would not be the first time, but I have the 
impression that some/all? of you are arguing for a different philosophy 
around R CMD check and Suggests/Depends. But the current design is not 
broken, it is working the way it has been advertised for many years now. 
It provides a fine-grained dependency graph for end users, not 
developers and testers. Being able to suggest packages for use in 
testing, when they are not needed for regular use is a good thing. A 
package failing R CMD check when the suggested packages are not 
available is not a bug, it is a feature following the rules as they have 
been designed. If you want to check a package then you need to install 
things that are needed to check it. If R CMD check skipped testing 
because suggested packages were not available then you will have many 
packages not being tested properly, that is, lots of broken packages 
passing R CMD check. (I've done this to myself sometimes using 
if(require()).) There are situations where some testing needs to be 
skipped, for example, license requirements and special databases, but 
this needs to be done carefully, and my impression is that if(require()) 
provides most of what is necessary, sometimes along with environment 
variables. Perhaps this is not elegant, but it does work and is not 
difficult.

            

      
      
      
    

        
But how do you decide pass/fail when you do this? I think it will only 
pass when all the suggested packages are available?

Paul Gilbert