Skip to content

[R-pkg-devel] help with ASAN

7 messages · Steven Scott, Dirk Eddelbuettel, Rolf Turner

Original
Steven Scott
# 
I'm trying to build an ASAN enabled version of my R library to help debug
errors found by CRAN on my last submission.  I'm tantalizingly close, but
need some help figuring out what's wrong with my configuration.

I'm using a docker container that I think contains a version of R built
with the right options.  I launch the docker container as so:

docker run --cap-add SYS_PTRACE -e PASSWORD=<redacted> --rm -p 8787:8787 -v
/home/steve/code/BOOM:/home/steve/code/BOOM -it rocker/r-devel-ubsan-clang
/bin/bash

In this container I have added a ~/.R/Makevars file with the following
content:
CC= clang -std=c99 -fsanitize=address -fno-omit-frame-pointer
CXX= clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer
CXX11= clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer
FC = gfortran -fsanitize=address

When I build my package I get:
(... expected compiler output truncated except for final compile and link
step ... )
clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer -std=gnu++11
-I"/usr/share/R/include" -DNDEBUG -I`/usr/lib/R/bin/Rscript -e
"cat(system.file(package='Boom'))"`/include -DADD_ -DR_NO_REMAP -DRLANGUAGE
-I"/usr/local/lib/R/site-library/Boom/include"   -fpic  -g -O2
-fdebug-prefix-map=/build/r-base-MP6Q4u/r-base-3.6.2=.
-fstack-protector-strong -Wformat -Werror=format-security -Wdate-time
-D_FORTIFY_SOURCE=2 -g  -c splines.cc -o splines.o
clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer -std=gnu++11
-shared -L/usr/lib/R/lib -Wl,-z,relro -o BoomSpikeSlab.so
boom_spike_slab_init.o logit_spike_slab_wrapper.o mlm_spike_slab_wrapper.o
nested_regression_wrapper.o nnet_wrapper.o poisson_spike_slab_wrapper.o
probit_spike_slab_wrapper.o quantile_spike_wrapper.o
shrinkage_regression_wrapper.o spike_slab_wrapper.o splines.o
/usr/local/lib/R/site-library/Boom/lib/libboom.a -fsanitize=address
-L/usr/lib/R/lib -lR
installing to
/usr/local/lib/R/site-library/00LOCK-BoomSpikeSlab/00new/BoomSpikeSlab/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ?BoomSpikeSlab? in
dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object
'/usr/local/lib/R/site-library/00LOCK-BoomSpikeSlab/00new/BoomSpikeSlab/libs/BoomSpikeSlab.so':

/usr/local/lib/R/site-library/00LOCK-BoomSpikeSlab/00new/BoomSpikeSlab/libs/BoomSpikeSlab.so:
undefined symbol: __asan_option_detect_stack_use_after_return
Error: loading failed
Execution halted
ERROR: loading failed
* removing ?/usr/local/lib/R/site-library/BoomSpikeSlab?
* restoring previous ?/usr/local/lib/R/site-library/BoomSpikeSlab?
PACKAGE = BoomSpikeSlab
Dirk Eddelbuettel
# 
Steve,

Your message has me puzzled over a few things just like your previous message
from a few days ago.
On 25 March 2020 at 14:17, Steven Scott wrote:
| I'm trying to build an ASAN enabled version of my R library to help debug
| errors found by CRAN on my last submission.  I'm tantalizingly close, but
| need some help figuring out what's wrong with my configuration.
| 
| I'm using a docker container that I think contains a version of R built
| with the right options.  I launch the docker container as so:
| 
| docker run --cap-add SYS_PTRACE -e PASSWORD=<redacted> --rm -p 8787:8787 -v
| /home/steve/code/BOOM:/home/steve/code/BOOM -it rocker/r-devel-ubsan-clang
| /bin/bash

My goto alias for docker does a few of these but never the --cap-add (seems
like you may need it for profiling, likely not for asan/ubsan), the password,
or the port (but who knows, maybe you need them...).  So I'd say

 docker run --rm -v /home/steve/code/BOOM:/home/steve/code/BOOM \
    -it rocker/r-devel-ubsan-clang /bin/bash

[ For completeness I have this
  ## docker run and mount 
  alias dkrr='docker run --rm -ti -v$(pwd):/work -w /work

and would then just do

  dkrr rocker/r-devel-ubsan-clang bash

which has a lesser cognitive load. But I digress. ]

| In this container I have added a ~/.R/Makevars file with the following
| content:
| CC= clang -std=c99 -fsanitize=address -fno-omit-frame-pointer
| CXX= clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer
| CXX11= clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer
| FC = gfortran -fsanitize=address

For ASAN/UBSAN to work you should use the same setting as the compiled
'host'. As you benefit from a pre-made Docker container, its settings are

  CC="clang -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-omit-frame-pointer -fsanitize-address-use-after-scope" \
  CXX="clang++ -stdlib=libc++ -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-omit-frame-pointer -fsanitize-address-use-after-scope" \

per 

  https://github.com/rocker-org/r-devel-san-clang/blob/master/Dockerfile#L90-L91

so I recommend you use the same. I found all this mysterious enough back in
the day when I first set it up that I create a package with *known* triggers
for these detections so that I could be sure I had at least their detection
right.  It is still at CRAN:

  https://cran.r-project.org/package=sanitizers

These things to change with compiler releases and what not. If this Docker
container does not work, there is another very good one here

  https://github.com/wch/r-debug
  
| When I build my package I get:
| (... expected compiler output truncated except for final compile and link
| step ... )
| clang++ -std=c++11 -fsanitize=address -fno-omit-frame-pointer -std=gnu++11
[...]
| /usr/local/lib/R/site-library/00LOCK-BoomSpikeSlab/00new/BoomSpikeSlab/libs/BoomSpikeSlab.so:
| undefined symbol: __asan_option_detect_stack_use_after_return

Looks like you also need to link with the ASAN library, possibly via -lasan.
My Dockerfile referenced above has

  MAIN_LD="clang++ -stdlib=libc++ -fsanitize=undefined,address"

and it may be implied by the -fsanitize=... business.

Hth, Dirk
Dirk Eddelbuettel
# 
Steve,
On 25 March 2020 at 16:48, Dirk Eddelbuettel wrote:
| For ASAN/UBSAN to work you should use the same setting as the compiled
| 'host'. As you benefit from a pre-made Docker container, its settings are
| 
|   CC="clang -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-omit-frame-pointer -fsanitize-address-use-after-scope" \
|   CXX="clang++ -stdlib=libc++ -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-omit-frame-pointer -fsanitize-address-use-after-scope" \

Addendum / correction: Because R-devel in this container as been built with
these settings, your package does NOT need to repeat them in its src/Makevars.

Just invoke it via RD as in RD CMD INSTALL ... or RD CMD check ...

Hth, Dirk
Steven Scott
# 
Oohhh I didn't realize there was an RD.  That simplifies things a lot!
And for the record, I'm not surprised that my setup suggests that I don't
know what I'm doing viz-a-viz docker.  Everything was assembled by Google
search and spaghetti flinging, so I'm pleased to get some expert advice.

A grateful cheers and thank you for the help!
On Wed, Mar 25, 2020 at 5:21 PM Dirk Eddelbuettel <edd at debian.org> wrote:

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 26/03/20 10:17 am, Steven Scott wrote:

        

            

      
      
      
    

        
<SNIP>

*PLEASE*!!!  Your R *package*!!!

(A library is a *collection* of packages.)

Correct terminology *does* matter, in that incorrect terminology can on 
occasion lead to confusion that that can on occasion have unfortunate 
consequences.

cheers,

Rolf Turner

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Steven Scott
    

    

      
      #
    

  


  

      
Rolf, I will strive to do better.

Please note that you load a package with library(...).  Just sayin'.

        
On Wed, Mar 25, 2020 at 6:16 PM Rolf Turner <r.turner at auckland.ac.nz> wrote:

        

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 26/03/20 5:16 pm, Steven Scott wrote:

        

            

      
      
      
    

        
Yeah, this issue has arisen/been discussed (on R-help I think) in the 
past.  The rationale is that you use the library() function to "take a 
package out of a library" (like taking a book out of a library).  In 
retrospect the developers might wish they had used a different name for 
the library() function, but it's really too late to change anything now.

cheers,

Rolf