Skip to content

test suites for packages

8 messages · Whit Armstrong, Uwe Ligges, Brian G. Peterson +3 more

Original
Whit Armstrong
# 
Can anyone share some opinions on test suites for R packages?

I'm looking at testthat and RUnit. Does anyone have strong opinions on
either of those.

Any additional packages I should consider?

Thanks,
Whit
Uwe Ligges
#
On 17.05.2012 16:10, Whit Armstrong wrote:
Yes: R CMD check does the trick. See Writing R Extension and read about 
a package's test directory. I prefer frameworks that do not obfuscate 
failing test results on the CRAN check farm (as most other frameworks I 
have seen).

Best,
Uwe Ligges
Brian G. Peterson
#
On Thu, 2012-05-17 at 16:32 +0200, Uwe Ligges wrote:
Uwe:  I don't think that's completely fair.  RUnit and testthat tests
can be configured to be called from the R package tests directory, so
that they are run during R CMD check.

They don't *need* to be configured that way, so perhaps that's what
you're talking about.
Uwe Ligges
#
On 17.05.2012 16:52, Brian G. Peterson wrote:
I am talking about the problem that relevant output of test failures 
that may help to identify the problem is frequently not shown in the 
output of R CMD check when such frameworks are used - that is a major 
nuisance for CRAN automatisms.

If additional configuration steps are required, fine, but then package 
maintainers seem to forget about that step. We do not have the time to 
tell hundreds of package maintainers how to configure their preferred 
test framework (whatever it is).

Best,
Uwe
Matt Dowle
# 
Uwe Ligges <ligges <at> statistik.tu-dortmund.de> writes:
Not sure, but could it be that in some cases the output of test failures is 
there, but chopped off since CRAN displays the 13 line tail? At least that's 
what I've experienced, and reported, and asked to be increased in the past. 
Often the first error causes a cascade, so it's the head you need to see, not 
the tail. If I've got that right, how about a much larger limit than 13, say 
1000. Or the first 50 and last 50 lines of output.

Matthew
Uwe Ligges
#
On 17.05.2012 17:56, Matthew Dowle wrote:
R always reports the whole diffs of the tests.

Uwe
Cook, Malcolm
# 
svUnit - is Runit compatible and provides some  IDE integration and report
generation and easy syntax for defining tests.

I find it works a treat, and fits very nicely with my R coding/packaging
style (which also uses inlinedocs for easy package creation).

--Malcolm Cook
On 5/17/12 9:10 AM, "Whit Armstrong" <armstrong.whit at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    
5 days later

    

      
      

        


  

        

      Claudia Beleites
    

    

      
      #
    

  


  

      
I use svUnit, too.

I put a kind of standard skeleton into my packages that has a
packagename.unittest () function that will run all svUnit tests (if
svUnit is available). This function returns NA if svUnit is not
available, invisible (TRUE) if all tests are passed and  stops
otherwise. Which will cause R CMD check to fail.

My test directory contains one single tests.R file with the two lines:

library (packagename)
packagename.unittest ()


which gives me e.g.:

Running the tests in ?tests/tests.R? failed.
Last 13 lines of output:
    only logical matrix subscripts are allowed in replacement

  * :     ...) ... **ERROR**
  Error in `[<-.data.frame`(`*tmp*`, x.na, value = NA) :
    only logical matrix subscripts are allowed in replacement
                           kind timing                time unit msg
  test(kernelpls.fit)        OK  0.009 2012-05-23 14:38:49
  test(scale)                OK  0.009 2012-05-23 14:38:49
  test(.ldapreproc)   **ERROR**  0.007 2012-05-23 14:38:49
  test(pcalda)               OK  0.003 2012-05-23 14:38:49
  Error in errorLog(summarize = FALSE) : 0 failure(s) and 1 error(s)
  Calls: cbmodels.unittest -> errorLog
  Execution halted


While the 13 lines may not be enough if there are lots of tests, I can
easily run packagename.unittest () in an interactive session and start
tracking down the problem from there.


Claudia


Am 18.05.2012 17:28, schrieb Cook, Malcolm: