[xts, quantmod] segfault probelm when I work with memcpy function

W dniu 10 wrze?nia 2010 20:32 u?ytkownik Brian G. Peterson
<brian at braverock.com> napisa?:
Daniel,

I haven't tried your example, but I wonder why you aren't using C accessor
methods defined by xts itself or at least derived from the significant
amounts of C code in xts.

For example, your test code seems to bear a close resemblance in principle
to coredata.c, but you don't appear to have used or derived from that code.
I want to have a better expertise with SEXP and R and I hope in the
future I want to help to develop some R financial packages (xts, TTR
and blotter). So this is my main motivation.
I understand that your 'real' problem is likely different from your
contrived example, but it still seems that you should leverage the C code
already in xts where possible.
I study xts code :) I could write some indicator in C and build xts
object directly from C, so you have only .Call('myind',IBM) and it's
return xts object. On large data and when you need to call this
function with high frequency it's much more faster then
.xts(.Call('myind',Cl(IBM)),getAttrib(IBM,'index')).

regards,
daniel
Regards,

  - Brian

On 09/10/2010 12:27 PM, Daniel Cegie?ka wrote:
Hi,

I work with SEXP C code and with xts and quantmod packages. I try to
touch how xts internal works.

So we have R session and:

ls()
character(0)
getSymbols('AAPL')     # quantmod package
[1] "AAPL"
ls()
[1] "AAPL"
str(AAPL)
An 'xts' object from 2007-01-03 to 2010-09-09 containing:
  Data: num [1:929, 1:6] 86.3 84 85.8 86 86.5 ...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
  Indexed by objects of class: [Date] TZ:
  xts Attributes:
List of 2
 $ src    : chr "yahoo"
 $ updated: POSIXct[1:1], format: "2010-09-10 18:42:10"
tail(AAPL,5)
           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900
 252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100
 258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200
 257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000
 262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700
 263.07
q()
Save workspace image? [y/n/c]: y
[danice at entropy ~]$

It's looks, that xts data in mamory looks like this (Open=o,High=h,Low=l
etc):

ooooo(...)ohhhhh(...)hlllll(...)lccccc(...)cvvvvv(...)vaaaaa(...)a

So if I want to read Open price I need to read this array (from C
code) from x[0] to x[nrow(x)-1] where x is the pointer do AAPL.

I have to test functions - one based on memcpy and second based on for
loop:

#include<R.h>
#include<Rinternals.h>

SEXP open(SEXP x) {
        int nr=nrows(x);
        SEXP r;
        PROTECT(r=allocVector(REALSXP,nr));

        memcpy(&REAL(r)[0],&REAL(x)[0],nr*sizeof(double));

        UNPROTECT(1);
        return(r);
}

SEXP open2(SEXP x) {
        int P=0;
        if (TYPEOF(x) != REALSXP) { PROTECT(x = coerceVector(x,REALSXP));
P++; }
        double *d_x = REAL(x);
        int nr = nrows(x);

        SEXP s;
        PROTECT(s = allocVector(REALSXP,nr));
        P++;
        double *d_s = REAL(s);

        int i;
        for (i=0;i<nr;i++) d_s[i] = d_x[i];

        UNPROTECT(P);
        return(s);
}

We starts R session again and:

# we have AAPL data in memory

ls()
[1] "AAPL"
tail(AAPL)
           AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume
AAPL.Adjusted
2010-09-02    251.26    252.17   248.57     252.17    14811900
 252.17
2010-09-03    255.09    258.78   254.50     258.77    18580100
 258.77
2010-09-07    256.64    259.53   256.25     257.81    12234200
 257.81
2010-09-08    259.78    264.39   259.10     262.92    18777000
 262.92
2010-09-09    265.04    266.52   262.92     263.07    15642700
 263.07
# now we call do open2 function

.Call('open2',AAPL)
  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)

# and now call to open based on memcpy

.Call('open',AAPL)
  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14  85.73  86.68  87.11  87.11  86.30  86.43
84.86  86.23  84.12

(...)

AND HERE IS MY PROBLEM:

We download new data:

getSymbols('IBM')
[1] "IBM"
####    WE DOWNLOAD NEW DATA

getSymbols('IBM')
[1] "IBM"
# and try open2 function (based on for loop)

.Call('open2',AAPL)
  [1]  86.29  84.05  85.77  85.96  86.45  94.75  95.94  94.59  95.68
97.56  92.10  88.63  89.14

(...)

# now we try open function based on memcpy
# ... and we will have a segfault

.Call('open',AAPL)
 *** caught segfault ***
address 0x2, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3

I have this problem only if I download new data...

Do someone know how can I solve this memcpy problem? memcpy should be
much faster in this kind of area... and I want to write some C based
extensions for xts package.

Best regards,
daniel

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

[xts, quantmod] segfault probelm when I work with memcpy function

Thread (3 messages)