I read in the RcppParallel blurb "The code that you write within parallel workers should not call the R or Rcpp API in any fashion", which is admirably clear. However, it leaves me without threadsafe access to distribution functions (dpois, dbinom etc.). In practice, so far, these R API calls seem to work for me, but can they be trusted? Is there an alternative? (It seems this question must have arisen before, but I haven't found an answer) Murray -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180416/1b0f1e5e/attachment.html>
[Rcpp-devel] Distribution functions threadsafe in RcppParallel?
7 messages · Dirk Eddelbuettel, Murray Efford, JJ Allaire +2 more
On 16 April 2018 at 13:41, Murray Efford wrote:
| I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer)
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
Thanks. This is all happening inside a package for CRAN, so I would rather avoid more complexity and potential platform-dependence, but I also cannot afford for it to break unpredictably (or otherwise). Murray
On Mon, Apr 16, 2018 at 3:33 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 16 April 2018 at 13:41, Murray Efford wrote: | I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer) -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180416/23d66ecc/attachment.html>
Here it's recommended that you simply read the source code to figure out if memory allocations or stack checking are done: https://rcppcore.github.io/RcppParallel/#api_restrictions On Sun, Apr 15, 2018 at 11:56 PM, Murray Efford <murray.efford at gmail.com> wrote:
Thanks. This is all happening inside a package for CRAN, so I would rather avoid more complexity and potential platform-dependence, but I also cannot afford for it to break unpredictably (or otherwise). Murray On Mon, Apr 16, 2018 at 3:33 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 16 April 2018 at 13:41, Murray Efford wrote: | I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer) -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
_______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180416/8e11ce7d/attachment.html>
For your information, here is a header-only library for distribution functions: https://github.com/kthohr/stats The API seems to be designed to mimic the R API. Best, Yixuan 2018-04-15 23:56 GMT-04:00 Murray Efford <murray.efford at gmail.com>:
Thanks. This is all happening inside a package for CRAN, so I would rather avoid more complexity and potential platform-dependence, but I also cannot afford for it to break unpredictably (or otherwise). Murray On Mon, Apr 16, 2018 at 3:33 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 16 April 2018 at 13:41, Murray Efford wrote: | I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer) -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
_______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Yixuan Qiu <yixuanq at gmail.com <yixuan.qiu at cos.name>> Department of Statistics, Purdue University -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180416/466c1969/attachment.html>
Greetings and Salutations, There is also the boost header libraries that you can use: https://www.boost.org/doc/libs/1_67_0/doc/html/boost_random.html Gallery example: http://gallery.rcpp.org/articles/timing-normal-rngs/ If you need parallel draws, see the sitmo engine: https://github.com/coatless/sitmo Also, you may wish to consider the rTRNG library: https://github.com/miraisolutions/rTRNG Sincerely, JJB From: <rcpp-devel-bounces at lists.r-forge.r-project.org> on behalf of Yixuan Qiu <yixuanq at gmail.com> Date: Monday, April 16, 2018 at 11:53 AM To: Murray Efford <murray.efford at gmail.com> Cc: "rcpp-devel at lists.r-forge.r-project.org" <rcpp-devel at lists.r-forge.r-project.org> Subject: Re: [Rcpp-devel] Distribution functions threadsafe in RcppParallel? For your information, here is a header-only library for distribution functions: https://github.com/kthohr/stats The API seems to be designed to mimic the R API. Best, Yixuan 2018-04-15 23:56 GMT-04:00 Murray Efford <murray.efford at gmail.com<mailto:murray.efford at gmail.com>>: Thanks. This is all happening inside a package for CRAN, so I would rather avoid more complexity and potential platform-dependence, but I also cannot afford for it to break unpredictably (or otherwise). Murray
On Mon, Apr 16, 2018 at 3:33 PM, Dirk Eddelbuettel <edd at debian.org<mailto:edd at debian.org>> wrote:
On 16 April 2018 at 13:41, Murray Efford wrote:
| I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer) -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org<mailto:edd at debian.org> _______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org<mailto:Rcpp-devel at lists.r-forge.r-project.org> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Yixuan Qiu <yixuanq at gmail.com<mailto:yixuan.qiu at cos.name>> Department of Statistics, Purdue University -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180416/ec5ca7fd/attachment-0001.html>
Thanks for all these good suggestions. StatsLib as suggested by Yixuan seems especially straightforward. I may yet find that dpois and dbinom from R are threadsafe. Murray On Tue, Apr 17, 2018 at 5:34 AM, Balamuta, James Joseph <
balamut2 at illinois.edu> wrote:
Greetings and Salutations, There is also the boost header libraries that you can use: https://www.boost.org/doc/libs/1_67_0/doc/html/boost_random.html Gallery example: http://gallery.rcpp.org/articles/timing-normal-rngs/ If you need parallel draws, see the sitmo engine: https://github.com/coatless/sitmo Also, you may wish to consider the rTRNG library: https://github.com/ miraisolutions/rTRNG Sincerely, JJB *From: *<rcpp-devel-bounces at lists.r-forge.r-project.org> on behalf of Yixuan Qiu <yixuanq at gmail.com> *Date: *Monday, April 16, 2018 at 11:53 AM *To: *Murray Efford <murray.efford at gmail.com> *Cc: *"rcpp-devel at lists.r-forge.r-project.org" < rcpp-devel at lists.r-forge.r-project.org> *Subject: *Re: [Rcpp-devel] Distribution functions threadsafe in RcppParallel? For your information, here is a header-only library for distribution functions: https://github.com/kthohr/stats The API seems to be designed to mimic the R API. Best, Yixuan 2018-04-15 23:56 GMT-04:00 Murray Efford <murray.efford at gmail.com>: Thanks. This is all happening inside a package for CRAN, so I would rather avoid more complexity and potential platform-dependence, but I also cannot afford for it to break unpredictably (or otherwise). Murray On Mon, Apr 16, 2018 at 3:33 PM, Dirk Eddelbuettel <edd at debian.org> wrote: On 16 April 2018 at 13:41, Murray Efford wrote: | I read in the RcppParallel blurb "The code that you write within parallel | workers should not call the R or Rcpp API in any fashion", which is | admirably clear. However, it leaves me without threadsafe access to | distribution functions (dpois, dbinom etc.). In practice, so far, these R | API calls seem to work for me, but can they be trusted? Is there an | alternative? That's a fair question. They may work, as they are also exposed / available via the standalone R math library (see Writing R Extensions). As such, they may not required memory allocations or other interactions with the R process and hence "not call R ... in any fashion" per the above. But we can't say for sure. If you want to be safe, maybe stick to equivalent functions from a non-R source: C++11, Boost, ... Dirk | (It seems this question must have arisen before, but I haven't found an | answer) -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
_______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Yixuan Qiu <yixuanq at gmail.com <yixuan.qiu at cos.name>> Department of Statistics, Purdue University
-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20180417/9e62f6f3/attachment-0001.html>