[Rcpp-devel] RCppParallel Example Crashing R (Cross-Post from SO)

I posted the following to SO earlier this morning and it was suggested
that I repost this question to the rcpp-devel list for exposure.  For
reference, the original posting is at
http://stackoverflow.com/questions/27089888/rcppparallel-example-crashing-r
.

I have been trying to parallelize one of my Rcpp routines.  In doing so I
have been trying to follow the [Parallel Distance Calculation](
http://gallery.rcpp.org/articles/parallel-distance-matrix/) example from
jjalaire.  Unfortunately, once I got everything coded up and started to
play around, my R session would crash.  Sometimes after the first
execution, sometimes after the third.  To be honest, it was a crap shoot as
to when R would crash when I ran the routine.  So, I have paired down my
code to a small reproducible example to play with.

***Rcpp File (mytest.cpp)***

------------------------------------------------------------------------------------------------------------
    #include <Rcpp.h>
    // [[Rcpp::depends(RcppParallel)]]
    #include <RcppParallel.h>

    using namespace std;
    using namespace Rcpp;
    using namespace RcppParallel;

    struct MyThing : public Worker {
      RVector<double> _pc;
      RVector<double> _pcsd;

      MyThing(Rcpp::NumericVector _pc, Rcpp::NumericVector _pcsd) :
_pc(_pc), _pcsd(_pcsd){}

      void operator()(std::size_t begin, std::size_t end) {

        for(int j = begin; j <= end; j++) {
          _pc[j] = 1;
    //      _pcsd[j] = 1;
        }
      }
    };

    // [[Rcpp::export]]
    void calculateMyThingParallel() {

      NumericVector _pc(100);
      NumericVector _pcsd(100);

      MyThing mt(_pc, _pcsd);

      parallelFor(0, 100, mt);
    }

------------------------------------------------------------------------------------------------------------


***R Compilation and Execution Script (mytest.R)***

------------------------------------------------------------------------------------------------------------
    library(Rcpp)
    library(inline)

    sourceCpp('mytest.cpp')

    testmything = function() {
      calculateMyThingParallel()
    }

    if(TRUE) {
      for(i in 1:20) {
        testmything()
      }
    }

------------------------------------------------------------------------------------------------------------

The error seems to be directly related to my setting of the _pc and _pcsd
variables in the `operator()` method.  If I take those out things
dramatically improve.  Based on the Parallel Distance Calculation example,
I am not sure what it is that I have done wrong here.  I was under the
impression that RVector<type> was thread safe.  Although that is my
impression, I know this is an issue with threads somehow.  Can anybody help
me to understand why the above code randomly crashes my R sessions?

For information I am running the following:

  - Windows 7
  - R: 3.1.2
  - Rtools: 3.1
  - Rcpp: 0.11.3
  - inline: 0.3.13
  - RStudio: 0.99.62

Hello Justace,
It seems that you should write down j < end rather than j <= end.


Best,
Yixuan

2014-11-23 21:10 GMT-05:00 Justace Clutter <justacec at gmail.com>:

I posted the following to SO earlier this morning and it was suggested
that I repost this question to the rcpp-devel list for exposure.  For
reference, the original posting is at
http://stackoverflow.com/questions/27089888/rcppparallel-example-crashing-r
.

I have been trying to parallelize one of my Rcpp routines.  In doing so I
have been trying to follow the [Parallel Distance Calculation](
http://gallery.rcpp.org/articles/parallel-distance-matrix/) example from
jjalaire.  Unfortunately, once I got everything coded up and started to
play around, my R session would crash.  Sometimes after the first
execution, sometimes after the third.  To be honest, it was a crap shoot as
to when R would crash when I ran the routine.  So, I have paired down my
code to a small reproducible example to play with.

***Rcpp File (mytest.cpp)***

------------------------------------------------------------------------------------------------------------
    #include <Rcpp.h>
    // [[Rcpp::depends(RcppParallel)]]
    #include <RcppParallel.h>

    using namespace std;
    using namespace Rcpp;
    using namespace RcppParallel;

    struct MyThing : public Worker {
      RVector<double> _pc;
      RVector<double> _pcsd;

      MyThing(Rcpp::NumericVector _pc, Rcpp::NumericVector _pcsd) :
_pc(_pc), _pcsd(_pcsd){}

      void operator()(std::size_t begin, std::size_t end) {

        for(int j = begin; j <= end; j++) {
          _pc[j] = 1;
    //      _pcsd[j] = 1;
        }
      }
    };

    // [[Rcpp::export]]
    void calculateMyThingParallel() {

      NumericVector _pc(100);
      NumericVector _pcsd(100);

      MyThing mt(_pc, _pcsd);

      parallelFor(0, 100, mt);
    }

------------------------------------------------------------------------------------------------------------


***R Compilation and Execution Script (mytest.R)***

------------------------------------------------------------------------------------------------------------
    library(Rcpp)
    library(inline)

    sourceCpp('mytest.cpp')

    testmything = function() {
      calculateMyThingParallel()
    }

    if(TRUE) {
      for(i in 1:20) {
        testmything()
      }
    }

------------------------------------------------------------------------------------------------------------

The error seems to be directly related to my setting of the _pc and _pcsd
variables in the `operator()` method.  If I take those out things
dramatically improve.  Based on the Parallel Distance Calculation example,
I am not sure what it is that I have done wrong here.  I was under the
impression that RVector<type> was thread safe.  Although that is my
impression, I know this is an issue with threads somehow.  Can anybody help
me to understand why the above code randomly crashes my R sessions?

For information I am running the following:

  - Windows 7
  - R: 3.1.2
  - Rtools: 3.1
  - Rcpp: 0.11.3
  - inline: 0.3.13
  - RStudio: 0.99.62

--
Yixuan Qiu <yixuan.qiu at cos.name>
Department of Statistics,
Purdue University

I would say that was it...  Thanks for catching that.  Sometimes the
smallest thing can get you.  I have updated the SO question with the answer.

Now that I got this working I am seeing great speed improvements:

b1 = R native code
b2 = Rcpp non parallel
b3 = Rcpp parallel

  test replications elapsed relative user.self sys.self user.child
sys.child
1 b1()           10 1760.65   78.776   1753.76     0.56         NA
 NA
2 b2()           10  128.78    5.762    128.38     0.05         NA
 NA
3 b3()           10   22.35    1.000    170.68     0.06         NA
 NA


On Sun, Nov 23, 2014 at 9:15 PM, Yixuan Qiu <yixuan.qiu at cos.name> wrote:

Hello Justace,
It seems that you should write down j < end rather than j <= end.


Best,
Yixuan

2014-11-23 21:10 GMT-05:00 Justace Clutter <justacec at gmail.com>:

I posted the following to SO earlier this morning and it was suggested
that I repost this question to the rcpp-devel list for exposure.  For
reference, the original posting is at
http://stackoverflow.com/questions/27089888/rcppparallel-example-crashing-r
.

I have been trying to parallelize one of my Rcpp routines.  In doing so
I have been trying to follow the [Parallel Distance Calculation](
http://gallery.rcpp.org/articles/parallel-distance-matrix/) example
from jjalaire.  Unfortunately, once I got everything coded up and started
to play around, my R session would crash.  Sometimes after the first
execution, sometimes after the third.  To be honest, it was a crap shoot as
to when R would crash when I ran the routine.  So, I have paired down my
code to a small reproducible example to play with.

***Rcpp File (mytest.cpp)***

------------------------------------------------------------------------------------------------------------
    #include <Rcpp.h>
    // [[Rcpp::depends(RcppParallel)]]
    #include <RcppParallel.h>

    using namespace std;
    using namespace Rcpp;
    using namespace RcppParallel;

    struct MyThing : public Worker {
      RVector<double> _pc;
      RVector<double> _pcsd;

      MyThing(Rcpp::NumericVector _pc, Rcpp::NumericVector _pcsd) :
_pc(_pc), _pcsd(_pcsd){}

      void operator()(std::size_t begin, std::size_t end) {

        for(int j = begin; j <= end; j++) {
          _pc[j] = 1;
    //      _pcsd[j] = 1;
        }
      }
    };

    // [[Rcpp::export]]
    void calculateMyThingParallel() {

      NumericVector _pc(100);
      NumericVector _pcsd(100);

      MyThing mt(_pc, _pcsd);

      parallelFor(0, 100, mt);
    }

------------------------------------------------------------------------------------------------------------


***R Compilation and Execution Script (mytest.R)***

------------------------------------------------------------------------------------------------------------
    library(Rcpp)
    library(inline)

    sourceCpp('mytest.cpp')

    testmything = function() {
      calculateMyThingParallel()
    }

    if(TRUE) {
      for(i in 1:20) {
        testmything()
      }
    }

------------------------------------------------------------------------------------------------------------

The error seems to be directly related to my setting of the _pc and
_pcsd variables in the `operator()` method.  If I take those out things
dramatically improve.  Based on the Parallel Distance Calculation example,
I am not sure what it is that I have done wrong here.  I was under the
impression that RVector<type> was thread safe.  Although that is my
impression, I know this is an issue with threads somehow.  Can anybody help
me to understand why the above code randomly crashes my R sessions?

For information I am running the following:

  - Windows 7
  - R: 3.1.2
  - Rtools: 3.1
  - Rcpp: 0.11.3
  - inline: 0.3.13
  - RStudio: 0.99.62

--
Yixuan Qiu <yixuan.qiu at cos.name>
Department of Statistics,
Purdue University