Skip to content

R packages and test code

3 messages · Martin Maechler, A.J. Rossini

Original
A.J. Rossini
# 
Where is the best place to put evaluation/test code in creating an R
package?  I thought about a test subdirectory, but might it be better
to put in the R directory?  I guess I'm more concerned about the
output used for evaluating the success of the test, rather than the
test code and data, itself.

best,
-tony
Martin Maechler
# 
tony> Where is the best place to put evaluation/test code in creating
    tony> an R package?  I thought about a test subdirectory, but might it
    tony> be better to put in the R directory?  I guess I'm more concerned
    tony> about the output used for evaluating the success of the test,
    tony> rather than the test code and data, itself.

Good question.  
This hasn't been done yet in a formal general way.
MASS comes with the  scripts subdirectory, 
but there's no "make check" and comparison of desired and actual output.

1) "THE" basic test is running all the examples from all the *.Rd files.
  You can do this with contributed packages by

       R CMD check [options] [-l lib] pkg_1 ... pkg_n

  However, this just runs the examples 
	   (those *not* within a \dontrun{.} clause)
  and produces an  <pkg>-Ex.Rout  file.
  You still have to check yourself if the *.Rout file has the proper output

2) I assume the thing you desire is something like 
   what the distributed R does in  ./tests/  where there are
   foobar.R  and foobar.Rout.save  files 
	     for (foobar in c("is-things", "simple-tests", "eval-etc",
			      ...)) :
      R --vanilla < foobar.R > foobar.Rout
   is run 
   and a somewhat "tolerant" diff between  foobar.Rout and foobar.Rout.save
   is made. "SUCCESS" (actually `Ok') is reported when there are no
   (nontrivial) diffs.

   Try  "make check" with your R distribution if you don't know what I mean.

It shouldn't be hard to add this to the  "R INSTALL" scripts  :

   If there's  <pkg-src>/tests/
   for all pairs foobar.R and foobar.Rout.save (in that directory) 
   do the above test.

Tony, could you try to add this feature (to the relevant R scripts)
yourself?  or anyone else?
[it would be nice of course, if the identical scripts were used for 
 "base R"'s  ``make check''  and for the contributed packages's  
 "R CMD check .. pkg" or (preferably) "R INSTALL pkg ..."
]

Martin



-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
A.J. Rossini
# 
tony> Where is the best place to put evaluation/test code in
    tony> creating an R package?  I thought about a test subdirectory,
    tony> but might it be better to put in the R directory?  I guess
    tony> I'm more concerned about the output used for evaluating the
    tony> success of the test, rather than the test code and data,
    tony> itself.

    MM> Good question.  This hasn't been done yet in a formal general
    MM> way.  MASS comes with the scripts subdirectory, but there's no
    MM> "make check" and comparison of desired and actual output.

Is there a more formal description of the packaging system than that
given in the R-faq (which needs a bit of an update for ESS, Kurt)?

(I'd like to see one, if so, in order to specify exactly how to
consider the general situation).


    MM> 2) I assume the thing you desire is something like
    MM>    what the distributed R does in ./tests/ where there are
    MM>    foobar.R and foobar.Rout.save files
    MM> 	     for (foobar in c("is-things", "simple-tests",
    MM> 	     "eval-etc",
    MM> 			      ...)) :
    MM>       R --vanilla < foobar.R > foobar.Rout
    MM>    is run and a somewhat "tolerant" diff between foobar.Rout
    MM>    and foobar.Rout.save is made. "SUCCESS" (actually `Ok') is
    MM>    reported when there are no (nontrivial) diffs.

Yes, but not quite as formal as checking for non-trivial diffs.  I'd
be satisfied with just being able to construct a diff (or the new
file, to compare with the gold standard, for the diff) and making the
user visually go over them.

    MM> Tony, could you try to add this feature (to the relevant R
    MM> scripts) yourself?  or anyone else?  [it would be nice of
    MM> course, if the identical scripts were used for
    MM>  "base R"'s ``make check'' and for the contributed packages's
    MM>  "R CMD check .. pkg" or (preferably) "R INSTALL pkg ..."
    MM> ]

I can try, since I really want this (the tests, and quite possibly the
packaging specification for my next stat language project :-).
Probably won't happen until after my Vienna/Berlin/Atlanta trips over
the next 2+ weeks...

best,
-tony