I'm new to Rcpp and RcppArmadillo---so far I like it a lot! Thanks to the developers for their good work.
I run into a peculiar kind of problem when I try to pass a "large" Armadillo object back to R. Here's some code to replicate the problem in stylized form:
extern "C" SEXP testFun(SEXP L_)
{
const long L = Rcpp::as<long>(L_);
arma::mat X(L,1);
arma::mat Y(L,1);
X.fill(1);
Y.fill(2);
return Rcpp::List::create(
Rcpp::Named("X") = Rcpp::wrap(X),
Rcpp::Named("Y") = Rcpp::wrap(Y)
);
}
I compile (both using g++ and the Intel Compiler --- choice of compiler makes no difference) and then call from R:
L = 1000000
ret = .Call("testFun", as.integer(L))
print(ret$X[1:5,])
print(ret$Y[1:5,])
Here's what I often get as output:
[1] 1 1 1 1 1
[1] 1 1 1 1 1
However, the second rows should be all 2's!
When I try to pass smaller matrices, for example by setting L=100000, the problem goes away:
[1] 1 1 1 1 1
[1] 2 2 2 2 2
Also, the problem does not arise when I create and pass pack objects of type Rcpp::NumericMatrix; so far I've see the problem only with Armadillo objects. I've encountered this on two Macs running OS X 10.6.6, R 2.12.2, and I'm using the latest versions of Rcpp and RcppArmadillo.
Help on this matter is appreciated!
G?nter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/rcpp-devel/attachments/20110307/e3ef1a9b/attachment.htm>
[Rcpp-devel] Problem passing Armadillo objects back to R
4 messages · "Günter J. Hitsch", Dirk Eddelbuettel, Baptiste Auguie
Hi Guenter,
On 7 March 2011 at 16:03, "G?nter J. Hitsch" wrote:
|
| I'm new to Rcpp and RcppArmadillo---so far I like it a lot! Thanks to the
| developers for their good work.
|
| I run into a peculiar kind of problem when I try to pass a "large" Armadillo
| object back to R. Here's some code to replicate the problem in stylized form:
|
|
| extern "C" SEXP testFun(SEXP L_)
| {
| const long L = Rcpp::as<long>(L_);
| arma::mat X(L,1);
| arma::mat Y(L,1);
| X.fill(1);
| Y.fill(2);
| return Rcpp::List::create(
| Rcpp::Named("X") = Rcpp::wrap(X),
| Rcpp::Named("Y") = Rcpp::wrap(Y)
| );
| }
|
|
| I compile (both using g++ and the Intel Compiler --- choice of compiler makes
| no difference) and then call from R:
|
|
| L = 1000000
| ret = .Call("testFun", as.integer(L))
| print(ret$X[1:5,])
| print(ret$Y[1:5,])
|
|
| Here's what I often get as output:
|
| [1] 1 1 1 1 1
| [1] 1 1 1 1 1
|
| However, the second rows should be all 2's!
|
| When I try to pass smaller matrices, for example by setting L=100000, the
| problem goes away:
|
| [1] 1 1 1 1 1
| [1] 2 2 2 2 2
|
|
| Also, the problem does not arise when I create and pass pack objects of type
| Rcpp::NumericMatrix; so far I've see the problem only with Armadillo objects.
| I've encountered this on two Macs running OS X 10.6.6, R 2.12.2, and I'm using
| the latest versions of Rcpp and RcppArmadillo.
I can't replicate that. I agree with you that we may have an issue here as
for 'large' N (such as 500,000) I get a segfault which is definitely
wrong (PS: but see below). On the other hand, for smaller values it works:
R> require(inline)
Loading required package: inline
R>
R> src <- '
+ const long L = Rcpp::as<long>(ls);
+ arma::mat X(L,1);
+ arma::mat Y(L,1);
+ X.fill(1);
+ Y.fill(2);
+ return Rcpp::List::create(Rcpp::Named("X") = Rcpp::wrap(X),
+ Rcpp::Named("Y") = Rcpp::wrap(Y));'
R>
R> fun <- cxxfunction(signature(ls="numeric"), body=src, plugin="RcppArmadillo")
R>
R> head(as.data.frame(fun(5)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
R> head(as.data.frame(fun(500)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(5000)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(50000)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R>
I quickly followed one hunch and the segfault is due to the long value for
the index. If you use an int then even my (modest) 6gb ram machine is happy to
allocate 50,000,000 objects. I stopped at that size.
R> require(inline)
Loading required package: inline
R>
R> src <- '
+ const int L = Rcpp::as<int>(ls);
+ arma::mat X(L,1);
+ arma::mat Y(L,1);
+ X.fill(1);
+ Y.fill(2);
+ return Rcpp::List::create(Rcpp::Named("X") = Rcpp::wrap(X),
+ Rcpp::Named("Y") = Rcpp::wrap(Y));'
R>
R> fun <- cxxfunction(signature(ls="numeric"), body=src, plugin="RcppArmadillo")
R>
R> head(as.data.frame(fun(50000)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(250000)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(500000)))
X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(5000000)))
X Y
1 1 1
2 1 1
3 1 1
4 1 1
5 1 1
6 1 1
R> head(as.data.frame(fun(50000000)))
X Y
1 1 1
2 1 1
3 1 1
4 1 1
5 1 1
6 1 1
R>
Dirk
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
Hi, I get the same incorrect results as G?nter for both the <long> and <int> versions. Cheers, baptiste sessionInfo() R version 2.12.2 Patched (2011-03-02 r54645) Platform: i386-apple-darwin9.8.0/i386 (32-bit) locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets grid methods [8] base other attached packages: [1] RcppArmadillo_0.2.15 Rcpp_0.9.2 inline_0.3.8 [4] ggplot2_0.8.9 proto_0.3-8 reshape_0.8.3 [7] plyr_1.4 loaded via a namespace (and not attached): [1] tools_2.12.2
On 8 March 2011 14:41, Dirk Eddelbuettel <edd at debian.org> wrote:
Hi Guenter,
On 7 March 2011 at 16:03, "G?nter J. Hitsch" wrote:
|
| I'm new to Rcpp and RcppArmadillo---so far I like it a lot! ?Thanks to the
| developers for their good work.
|
| I run into a peculiar kind of problem when I try to pass a "large" Armadillo
| object back to R. ?Here's some code to replicate the problem in stylized form:
|
|
| extern "C" SEXP testFun(SEXP L_)
| {
| const long L ?= Rcpp::as<long>(L_);
| arma::mat X(L,1);
| arma::mat Y(L,1);
| X.fill(1);
| Y.fill(2);
| return Rcpp::List::create(
| Rcpp::Named("X") = Rcpp::wrap(X),
| Rcpp::Named("Y") = Rcpp::wrap(Y)
| );
| }
|
|
| I compile (both using g++ and the Intel Compiler --- choice of compiler makes
| no difference) and then call from R:
|
|
| L = 1000000
| ret ?= .Call("testFun", as.integer(L))
| print(ret$X[1:5,])
| print(ret$Y[1:5,])
|
|
| Here's what I often get as output:
|
| [1] 1 1 1 1 1
| [1] 1 1 1 1 1
|
| However, the second rows should be all 2's!
|
| When I try to pass smaller matrices, for example by setting L=100000, the
| problem goes away:
|
| [1] 1 1 1 1 1
| [1] 2 2 2 2 2
|
|
| Also, the problem does not arise when I create and pass pack objects of type
| Rcpp::NumericMatrix; ?so far I've see the problem only with Armadillo objects.
| ?I've encountered this on two Macs running OS X 10.6.6, R 2.12.2, and I'm using
| the latest versions of Rcpp and RcppArmadillo.
I can't replicate that. I agree with you that we may have an issue here as
for 'large' N (such as 500,000) I get a segfault which is definitely
wrong (PS: but see below). On the other hand, for smaller values it works:
R> require(inline)
Loading required package: inline
R>
R> src <- '
+ const long L ?= Rcpp::as<long>(ls);
+ arma::mat X(L,1);
+ arma::mat Y(L,1);
+ X.fill(1);
+ Y.fill(2);
+ return Rcpp::List::create(Rcpp::Named("X") = Rcpp::wrap(X),
+ ? ? ? ? ? ? ? ? ? ? ? ? ? Rcpp::Named("Y") = Rcpp::wrap(Y));'
R>
R> fun <- cxxfunction(signature(ls="numeric"), body=src, plugin="RcppArmadillo")
R>
R> head(as.data.frame(fun(5)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
R> head(as.data.frame(fun(500)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(5000)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(50000)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R>
I quickly followed one hunch and the segfault is due to the long value for
the index. If you use an int then even my (modest) 6gb ram machine is happy to
allocate 50,000,000 objects. I stopped at that size.
R> require(inline)
Loading required package: inline
R>
R> src <- '
+ const int L ?= Rcpp::as<int>(ls);
+ arma::mat X(L,1);
+ arma::mat Y(L,1);
+ X.fill(1);
+ Y.fill(2);
+ return Rcpp::List::create(Rcpp::Named("X") = Rcpp::wrap(X),
+ ? ? ? ? ? ? ? ? ? ? ? ? ? Rcpp::Named("Y") = Rcpp::wrap(Y));'
R>
R> fun <- cxxfunction(signature(ls="numeric"), body=src, plugin="RcppArmadillo")
R>
R> head(as.data.frame(fun(50000)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(250000)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(500000)))
?X Y
1 1 2
2 1 2
3 1 2
4 1 2
5 1 2
6 1 2
R> head(as.data.frame(fun(5000000)))
?X Y
1 1 1
2 1 1
3 1 1
4 1 1
5 1 1
6 1 1
R> head(as.data.frame(fun(50000000)))
?X Y
1 1 1
2 1 1
3 1 1
4 1 1
5 1 1
6 1 1
R>
Dirk
--
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
_______________________________________________ Rcpp-devel mailing list Rcpp-devel at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
On 8 March 2011 at 14:50, baptiste auguie wrote:
| I get the same incorrect results as G?nter for both the <long> and | <int> versions. Ack, thanks for the confirmation. Maybe Romain has a chance to take a look as I do not have a Mac handy. Dirk
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com