Skip to content

[Bioc-devel] IntAE buffers needs to be freed or not??

5 messages · Hervé Pagès, Ge Tan

Original
Ge Tan
# 
Hi all,

I am using the IntAE buffers (taken from IRanges packages) in my .Call() code.
Sample code in C:

IntAE width_buf;
width_buf = new_IntAE(0, 0, 0);
for(?){
?? ?IntAE_insert_at(&width_buf, IntAE_get_nelt(&width_buf), width);
}
PROTECT(width = new_INTEGER_from_IntAE(&width_buf));
UNPROTECT(1);
return(width);

So after using .Call(), do I need to run something like ".Call("AEbufs_free", PACKAGE="IRanges")" in R?
I got this from the IRanges/src/AEbufs.c.

Thanks!
Ge
Hervé Pagès
# 
Hi Ge,

No you don't. At least not currently, because those buffers use
transient memory which will be freed automatically when .Call()
returns. However, I might change the implementation of the IntAE
buffers to use user-controlled memory at some point (malloc() is
so much faster than R_alloc(), about 10x-20x for me), so when this
happens you will need to do something like

   .Call("AEbufs_free", PACKAGE="IRanges")

unless you call your .Call entry point with .Call2 (defined in the
IRanges package), which will take care of doing that for you. I highly
recommend you do that if you use the IntAE buffers in your C code or
if you call C functions that use the IntAE buffers (and a lot of C
functions in IRanges and Biostrings use them).

Cheers,
H.
On 07/19/2013 03:35 AM, Ge Tan wrote:

  
    

      
      
    

  


  


      

    

    
2 days later

    

      
      

        


  

        

      Ge Tan
    

    

      
      #
    

  


  

      
Hi Herv?,

Thank you very much!
So how about Calloc? Does it run much slower than the direct malloc() too?

Best,
Ge

----------------------------------------

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
Hi Ge,

        
On 07/22/2013 02:59 AM, Ge Tan wrote:

            

      
      
      
    

        
Calloc() is just a wrapper to calloc() so it's also user-controlled
memory (like malloc()). The difference with malloc() is that it sets the
memory to zero so I expect it to be almost as fast as malloc(), and
therefore, also much faster than R_alloc().

Note that the auto-extending buffers (like IntAE) in IRanges don't need
to be set to zero so the choice is really between malloc/realloc (user-
controlled memory) and R_alloc/S_realloc (transient memory).

Finally I should be a little bit more precise. In addition to malloc()
being much faster than R_alloc(), realloc() is also much faster than
S_realloc(). What I observed (last time I checked) was that using
malloc/realloc was about 10x-20x faster (on my Linux system) than using
R_alloc/S_realloc for growing a buffer. The reason S_realloc() is much
slower than realloc() can easily be understood by looking at its
implementation (<R_home>/src/main/memory.c):

   char *S_realloc(char *p, long new, long old, int size)
   {
     size_t nold;
     char *q;
     /* shrinking is a no-op */
     if(new <= old) return p;
     q = R_alloc((size_t)new, size);
     nold = (size_t)old * size;
     memcpy(q, p, nold);
     memset(q + nold, 0, (size_t)new*size - nold);
     return q;
   }

It calls R_alloc(), then copies the data from the old place to the
new place, and then sets the remaining memory to zero.

OTOH realloc() is system-dependent and my understanding is that,
on a descent OS, it will try to do an in-place reallocation, thus
avoiding the need to move the data around.

Cheers,
H.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Ge Tan
    

    

      
      #
    

  


  

      
Hi Herv?,

Thank you! You always saved my life :-)

Best,
Ge


----------------------------------------