Skip to content

[R-pkg-devel] CRAN valgrind reports with zero leaks - actionable?

9 messages · Ivan Krylov, Dirk Eddelbuettel, Simon Urbanek +1 more

Original
Dirk Eddelbuettel
# 
[ Five days ago, I sent (most) of the message below to a CRAN team member. I
have not had a reply, and it now appears that we are asked to email here so I
am resending. ]

A package of mine shows a valgrind report at with the details clearly
indicating that no memory is lost: 

    ==3533933== LEAK SUMMARY:
    ==3533933==    definitely lost: 0 bytes in 0 blocks
    ==3533933==    indirectly lost: 0 bytes in 0 blocks
    ==3533933==      possibly lost: 0 bytes in 0 blocks

The continuous integration setup for the package includes a nightly job
against multiple version of the underlying C++ library to check just this.
In those job we do NOT trigger an alert when, as here, zero bytes are
reported lost.

Now, valgrind does report something possibly due (as the _Writing R
Extensions_ manual notes) to compiler optimisation. So we currently consider
this spurious.

Could CRAN let us know if this is considered a 'must fix' issue?  And if so,
maybe so not show the alarmingly red 'valgrind' link to a report with zero leaks?

Many thanks as always for all these thorough tests.

Sincerely,  Dirk
Ivan Krylov
# 
On Thu, 18 Aug 2022 06:55:01 -0500
Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        
Apologies if I'm misunderstanding the problem, but which package is it?
Does the ERROR SUMMARY also show 0 errors?

I would have assumed it's the dtts package, but that one shows a
non-zero "possibly lost" value. For dtts [*], there's also "invalid
read" messages intermixed with the test output, although I'm finding
them very confusing: how can it be invalid if it's within the bounds of
the allocation and the address seems to indicate correct alignment?

  
    
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 18 August 2022 at 15:53, Ivan Krylov wrote:

        
| Apologies if I'm misunderstanding the problem, but which package is it?

I do not believe it matters for the overall policy question I asked CRAN.

Dirk

  
    

      
      
    

  


  


      

    

    
6 days later

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
Dear CRAN Team,

        
On 18 August 2022 at 06:55, Dirk Eddelbuettel wrote:

        
| 
| [ Five days ago, I sent (most) of the message below to a CRAN team member. I
| have not had a reply, and it now appears that we are asked to email here so I
| am resending. ]

An answer would be much appreciated after the five initial days of this being
ignored as a private mail followed by a further six days of silence out here
in the open on this very list we once set up for help with packaging issues.

Can we ignore valgrind reports which show 'definitely lost: 0 bytes' ?

Curious,  Dirk


| A package of mine shows a valgrind report at with the details clearly
| indicating that no memory is lost: 
| 
|     ==3533933== LEAK SUMMARY:
|     ==3533933==    definitely lost: 0 bytes in 0 blocks
|     ==3533933==    indirectly lost: 0 bytes in 0 blocks
|     ==3533933==      possibly lost: 0 bytes in 0 blocks
| 
| The continuous integration setup for the package includes a nightly job
| against multiple version of the underlying C++ library to check just this.
| In those job we do NOT trigger an alert when, as here, zero bytes are
| reported lost.
| 
| Now, valgrind does report something possibly due (as the _Writing R
| Extensions_ manual notes) to compiler optimisation. So we currently consider
| this spurious.
| 
| Could CRAN let us know if this is considered a 'must fix' issue?  And if so,
| maybe so not show the alarmingly red 'valgrind' link to a report with zero leaks?
| 
| Many thanks as always for all these thorough tests.
| 
| Sincerely,  Dirk
| 
| 
| -- 
| dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
Dirk,

as Ivan pointed out there is no indication that your package was flagged due to 0 byte leaks given other possible problems in the report which is likely why it is red, but we can't know since you didn't say, so it's all purely speculative. Given the total lack of details I wouldn't expect any further answer here unless you can provide those.

BTW: if you want an answer from CRAN, you should write to cran at R-project.org (I have not seen any mention of it there) since the actually relevant member may not see it otherwise.

Cheers,
Simon

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
Simon,

What you attempt to answer is not the relevant question (as Ivan concurred in
private email).  I can only invite you again to maybe venture to answer the
question I asked rather than belabor a point I didn't make (even if you fancy
lecturing on that one).

Lastly, as you may may missed it in two previous emails: this public post was
preceded by a private email that was also ignored, hence no repeat to private
email to cran at r-project.org.

Cheers,  Dirk

        
On 25 August 2022 at 09:57, Simon Urbanek wrote:

        
| Dirk,
| 
| as Ivan pointed out there is no indication that your package was flagged due to 0 byte leaks given other possible problems in the report which is likely why it is red, but we can't know since you didn't say, so it's all purely speculative. Given the total lack of details I wouldn't expect any further answer here unless you can provide those.
| 
| BTW: if you want an answer from CRAN, you should write to cran at R-project.org (I have not seen any mention of it there) since the actually relevant member may not see it otherwise.
| 
| Cheers,
| Simon
| 
|

        
| > On 25/08/2022, at 8:12 AM, Dirk Eddelbuettel <edd at debian.org> wrote:

        
| > 
| > 
| > Dear CRAN Team,
| >

        
| > On 18 August 2022 at 06:55, Dirk Eddelbuettel wrote:

        
| > | 
| > | [ Five days ago, I sent (most) of the message below to a CRAN team member. I
| > | have not had a reply, and it now appears that we are asked to email here so I
| > | am resending. ]
| > 
| > An answer would be much appreciated after the five initial days of this being
| > ignored as a private mail followed by a further six days of silence out here
| > in the open on this very list we once set up for help with packaging issues.
| > 
| > Can we ignore valgrind reports which show 'definitely lost: 0 bytes' ?
| > 
| > Curious,  Dirk
| > 
| > 
| > | A package of mine shows a valgrind report at with the details clearly
| > | indicating that no memory is lost: 
| > | 
| > |     ==3533933== LEAK SUMMARY:
| > |     ==3533933==    definitely lost: 0 bytes in 0 blocks
| > |     ==3533933==    indirectly lost: 0 bytes in 0 blocks
| > |     ==3533933==      possibly lost: 0 bytes in 0 blocks
| > | 
| > | The continuous integration setup for the package includes a nightly job
| > | against multiple version of the underlying C++ library to check just this.
| > | In those job we do NOT trigger an alert when, as here, zero bytes are
| > | reported lost.
| > | 
| > | Now, valgrind does report something possibly due (as the _Writing R
| > | Extensions_ manual notes) to compiler optimisation. So we currently consider
| > | this spurious.
| > | 
| > | Could CRAN let us know if this is considered a 'must fix' issue?  And if so,
| > | maybe so not show the alarmingly red 'valgrind' link to a report with zero leaks?
| > | 
| > | Many thanks as always for all these thorough tests.
| > | 
| > | Sincerely,  Dirk
| > | 
| > | 
| > | -- 
| > | dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| > 
| > -- 
| > dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| > 
| > ______________________________________________
| > R-package-devel at r-project.org mailing list
| > https://stat.ethz.ch/mailman/listinfo/r-package-devel
| > 
|

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
As a follow-up:

1. The valgrind summary states 'zero leaks' (see my Subject) but hints at
   'errors' even when none are in the output. It suggests `valgrind -s` but
   when we run `R CMD check --use-valgrind` there is no means for setting
   such options. 

2. I once attempted what _Writing R Extensions_ suggests and created a file
   ~/.valgrindrc which contained `-s` but to no avail.

3. But _Writing R Extensions_ has another (newer ?) hint and using the
   command-line invocation of
     VALGRIND_OPTS="-s" R CMD check --use-valgrind package_verion.tar.gz
   did the trick. The resulting report is clear, the errors all come from
   a third-party system library I have no zero control over.

4. I will look into amending the nightly valgrind at GitHub Actions run to
   set this option to collect some more data.

5. The lack of any response from CRAN is very frustrating. 

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Uwe Ligges
    

    

      
      #
    

  


  

      
On 25.08.2022 00:12, Dirk Eddelbuettel wrote:

            

      
      
      
    

        
Can you please tell us the subject / date you used? I do not see it in 
our archive, but I may use the wrong search criteria.
(Note that we are searching in an archive of 70000 messages from/to 
CRAN/CRAN-submissions produced only in 2022, i.e. > 300 mail messages a 
day.)

Best,
Uwe

            

      
      
      
    

  
  


  


      

    

    
4 days later

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
Based on further (off-list) follow-ups with Simon and Uwe, the situation can
now be summarized as follows:

 - there appears to be no word about non-leak errors reported by valgrind in
   the official documentation; I will have to plead that those with the power
   to amend these documents maybe address that one day

 - it has been confirmed that when such a valgrind report is present, there
   is no auto-processing of the package and a manual re-check is done; as a
   package maintainer I rather avoid that to minimize manual interventions by
   both the CRAN team and the package maintainer (i.e. myself)

 - using `VALGRIND_OPTS="-s" R -d valgrind -f nameOfTheFile.R` is helpful,
   especially for the test framework I use where test files are standard R
   scripts; this helps in locating the issue

 - in the case that spawned this, the error actually comes from just one test
   and is tickled by a third-party system library so there is nothing for us
   to do here, besides not to run the test

 - so I moved the test into a testfile matching 'test_.*_extra.R' and now
   exclude files with that pattern via .Rbuildignore; that gives us a fuller
   test surface when testing locally via the full the test directory yet
   permits to isolate-away tests we would rather not be running at CRAN.

My thanks to everybody who helped with the question and clarified the context.

Dirk