Skip to content

[xts, quantmod] segfault probelm when I work with memcpy function

3 messages · Daniel Cegiełka, Brian G. Peterson

Original
Daniel Cegiełka
# 
Hi,

I work with SEXP C code and with xts and quantmod packages. I try to
touch how xts internal works.

So we have R session and:
character(0)
[1] "AAPL"
[1] "AAPL"
An ?xts? object from 2007-01-03 to 2010-09-09 containing:
  Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
  Indexed by objects of class: [Date] TZ:
  xts Attributes:
List of 2
 $ src    : chr "yahoo"
 $ updated: POSIXct[1:1], format: "2010-09-10 18:42:10"
AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900        252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100        258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200        257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000        262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700        263.07
Save workspace image? [y/n/c]: y
[danice at entropy ~]$


It's looks, that xts data in mamory looks like this (Open=o,High=h,Low=l etc):

ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a

So if I want to read Open price I need to read this array (from C
code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.


I have to test functions - one based on memcpy and second based on for loop:


#include <R.h>
#include <Rinternals.h>

SEXP open(SEXP x) {
        int nr=nrows(x);
        SEXP r;
        PROTECT(r=allocVector(REALSXP,nr));

        memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));

        UNPROTECT(1);
        return(r);
}


SEXP open2(SEXP x) {
        int P=0;
        if (TYPEOF(x) != REALSXP) { PROTECT(x = coerceVector(x,REALSXP)); P++; }
        double *d_x = REAL(x);
        int nr = nrows(x);

        SEXP s;
        PROTECT(s = allocVector(REALSXP,nr));
        P++;
        double *d_s = REAL(s);

        int i;
        for (i=0;i<nr;i++) d_s[i] = d_x[i];

        UNPROTECT(P);
        return(s);
}

We starts R session again and:
[1] "AAPL"
AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900        252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100        258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200        257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000        262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700        263.07
[1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)
[1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)



AND HERE IS MY PROBLEM:

We download new data:
[1] "IBM"
[1] "IBM"
[1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14

(...)
*** caught segfault ***
address 0x2, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3


I have this problem only if I download new data...

Do someone know how can I solve this memcpy problem? memcpy should be
much faster in this kind of area... and I want to write some C based
extensions for xts package.

Best regards,
daniel
Brian G. Peterson
# 
Daniel,

I haven't tried your example, but I wonder why you aren't using C 
accessor methods defined by xts itself or at least derived from the 
significant amounts of C code in xts.

For example, your test code seems to bear a close resemblance in 
principle to coredata.c, but you don't appear to have used or derived 
from that code.

I understand that your 'real' problem is likely different from your 
contrived example, but it still seems that you should leverage the C 
code already in xts where possible.

Regards,

    - Brian
On 09/10/2010 12:27 PM, Daniel Cegie?ka wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Daniel Cegiełka
    

    

      
      #
    

  


  

      
W dniu 10 wrze?nia 2010 20:32 u?ytkownik Brian G. Peterson
<brian at braverock.com> napisa?:

            

      
      
      
    

        
I want to have a better expertise with SEXP and R and I hope in the
future I want to help to develop some R financial packages (xts, TTR
and blotter). So this is my main motivation.

            

      
      
      
    

        
I study xts code :) I could write some indicator in C and build xts
object directly from C, so you have only .Call('myind',IBM) and it's
return xts object. On large data and when you need to call this
function with high frequency it's much more faster then
.xts(.Call('myind',Cl(IBM)),getAttrib(IBM,'index')).

regards,
daniel