[R-pkg-devel] CRAN pretest: Overall checktime 14 min > 10 min
On 21.05.2018 18:04, Dirk Eddelbuettel wrote:
On 21 May 2018 at 17:46, Uwe Ligges wrote: | In addition to what Dirk said, I just added this experimental test for | CRAN incoming checks few days ago and it should not reject but lead to | manual inspection, this will be fixed on CRAN side shortly. | | Nevertheless: The idea is that we have timeouts for checking a package | and we want to be alerted of future timeout problems in advance when a | package is submitted. Which is totally understandable given the limits we have on (human, chiefly) resources here -- but it is also worth reminding ourselves from time to time that this of course reduces actual test coverage ... which is generally not we want with these tests. This whole topic needs some more discussion.
A very difficult topic, while some authors do not supply any sensible checks nor evben examples and we have to force them to add even simplest code for some runtime checks, there are other who try too much. We are frequently bitten by cases where maintainers try to increase code coverage in the tests, but actually do test features of other packages (such as warning or error messages another package or R generates). One of the issues to be discussed would be that sensible testing is required, but not everythuing that leads to a larger coverage percentage makes sense ... Another frequent thing we see is thast people run MCMC simulations in the tests with thousands of iterations that are actually not needed for testing. Best, Uwe
On 21 May 2018 at 18:41, Chris Njuguna wrote: | Thank you for this! It makes a lot of sense. I will adopt this version | naming and testing scheme. Nice. I didn't give to much detail but if glance eg at https://github.com/RcppCore/Rcpp/blob/master/tests/doRUnit.R#L45-L52 for the detection (including an environment variable override) and eg at https://github.com/RcppCore/Rcpp/blob/master/inst/unitTests/runit.DataFrame.R#L21-L23 for one use case you should be able to get started. Ping me off-list with any questions. Dirk