Skip to content

[R-pkg-devel] What to do when you can't reproduce CRAN R CMD check unit test failures?

6 messages · Bob Rudis, Richard Cotton, Kevin Ushey +1 more

Original
Richard Cotton
# 
Yesterday I uploaded a new version of the pathological package to
CRAN.  It was initially accepted but today I got a message saying that
some of the tests for the decompose_path function are failing under
Windows.

I've tested the package on a local Windows 10 machine with R-3.3.0 and
R-devel, and in Win 7 and Win 8 compatibility modes, and on Windows
Server 2012 via AppVeyor.  In all cases, all the tests pass.

I tried the CRAN win builder service today, and again, all tests pass.
http://win-builder.r-project.org/0ZrX4I75nvzM/examples_and_tests/tests_i386/testthat.Rout
http://win-builder.r-project.org/0ZrX4I75nvzM/examples_and_tests/tests_x64/testthat.Rout

The failure portion of the output I got from Uwe is as follows:

* checking tests ... ERROR
Running the tests in 'tests/testthat.R' failed.
Last 13 lines of output:
  1. Failure: decompose_path handles paths with no directory and a
single extension in the filename. (@test_decompose_path.R#42)
  2. Failure: decompose_path handles paths with no directory and a
double extension in the filename. (@test_decompose_path.R#72)
  3. Failure: decompose_path handles paths with no directory and no
extension in the filename. (@test_decompose_path.R#103)
  4. Failure: decompose_path handles filenames containing a '.' and an
extension. (@test_decompose_path.R#133)
  5. Failure: decompose_path handles the current directory as '.'.
(@test_decompose_path.R#221)
  6. Failure: decompose_path handles the parent directory as '..'.
(@test_decompose_path.R#236)
  7. Failure: decompose_path handles files inside '.'.
(@test_decompose_path.R#251)
  8. Failure: decompose_path works with a character vector input.
(@test_decompose_path.R#321)
  9. Failure: decompose_path works with a factor input.
(@test_decompose_path.R#333)
  1. ...


Since this isn't hugely informative as to the problem, I'd like
general advice on how I can diagnose a failing test when I can't
reproduce it.

It would also be useful if anyone can run R CMD check and see if they
find test failures.

The CRAN release is here (the problematic version is 0.0-8):
https://cran.r-project.org/web/packages/pathological

and the github version is currently in the same state, available here:
https://github.com/richierocks/pathological
Bob Rudis
# 
did you try winbuilder R-devel? I occasionally (ok, often) forget to
do that and it sometimes kick out different errors than R-devel on
local systems.
On Wed, May 11, 2016 at 2:40 AM, Richard Cotton <richierocks at gmail.com> wrote:
Richard Cotton
# 
Thanks Bob,

I'd forgotten about the option to run win-builder against R-devel.

Though it seems that in this case, all the tests are passing there too.

http://win-builder.r-project.org/21kc9XGKdANF/examples_and_tests/tests_i386/testthat.Rout
http://win-builder.r-project.org/21kc9XGKdANF/examples_and_tests/tests_x64/testthat.Rout

I'm now completely out of ideas as to where the problem could be.
On 11 May 2016 at 15:46, boB Rudis <bob at rudis.net> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Kevin Ushey
    

    

      
      #
    

  


  

      
Is it possible that `getwd()` is reporting something on the CRAN build
servers that your `decompose_path()` doesn't handle? For example, your
tests fail for me if I run them while in the root directory (on OS X).

Kevin

        
On Thu, May 12, 2016 at 12:37 AM, Richard Cotton <richierocks at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Bob Rudis
    

    

      
      #
    

  


  

      
Of course the Great And Powerful Ushey did the super-smart thing and
actually tested the code locally. Apologies for not thinking of doing
the same on my end (though I'm also OS X):

    > getwd()
    [1] "/Users/bob/Development/pathological"

    > test()
    Loading pathological
    Loading required package: testthat
    Testing pathological
    ..................1..2..W...................................................................................................................
    Warnings -----------------------------------------------------------------------
    1. copy_dir works with recursive = TRUE (@test_dir_copy.R#34) -
The files ?javaconf?, ?ldpaths?, ?Makeconf?, ?Renviron?,
?repositories? were not copied successfully.

    Failed -------------------------------------------------------------------------
    1. Failure: decompose_path handles missing paths.
(@test_decompose_path.R#279) -
    `warnings` does not match "Coercing .+ to class
[[:punct:]]character[[:punct:]]\\.".
    Actual value: "Coercing x to class ?character?."


    2. Failure: decompose_path works with a factor input.
(@test_decompose_path.R#329)
    `warnings` does not match "Coercing .+ to class
[[:punct:]]character[[:punct:]]\\.".
    Actual value: "Coercing x to class ?character?."


    DONE ===========================================================================

I'm using R version 3.3.0 RC (2016-05-01 r70572) & testthat 1.0.2
(2016-04-23 CRAN (R 3.3.0)

I also get those same two errors on a slightly out-of-date R-devel on Ubuntu.

        
On Thu, May 12, 2016 at 2:09 AM, Kevin Ushey <kevinushey at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Martyn Plummer
    

    

      
      #
    

  


  

      
On Thu, 2016-05-12 at 01:09 -0500, Kevin Ushey wrote:

            

      
      
      
    

        
It does not work on a Windows shared drive:

            

      
      
      
    

        
[1] "\\\\vboxsrv/duchamp"

whereas:

            

      
      
      
    

        
??????????????????dirname filename extension
foo.tgz //vboxsrv/duchamp??????foo???????tgz

Martyn

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

      
      
    

        
This message and its attachments are strictly confidenti...{{dropped:8}}