Support for long arrays
With that much RAM, you probably have many cores. Consider splitting your long array across several R sessions with distributed parallel processing so that no one segment is over the limit. You will likely need the parallel processing anyway to make your computation time tolerable. See the pbdDEMO package and the rest of pbdR packages (see r-pbd.org). Cheers, George
On 1/6/14 6:00 AM, r-devel-request at r-project.org wrote:
---------------------------------------------------------------------- Message: 1 Date: Sun, 5 Jan 2014 20:18:06 +0000 From: Boris Aronshtam <baronshtam at lightminersystems.com> To: "r-devel at r-project.org" <r-devel at r-project.org> Cc: Boris Aronshtam <baronshtam at lightminersystems.com> Subject: [Rd] Support for long arrays Message-ID: <9D80368C6A00834189C1982FB64EFA3007EE9002 at ORD2MBX07C.mex05.mlsrvr.com> Content-Type: text/plain My machine has hundreds of GB of RAM. Is there a way to create an array having more than 2 Billion (2^31) rows? I am trying to create a large array using: d = c(1e+10,2) a=array(0,d) This results in the error: Error in array(0, d) : negative length vectors are not allowed In addition: Warning message: In array(0, d) : NAs introduced by coercion Note that I can create a vector of the same large size: v=c(1:2e+10) [[alternative HTML version deleted]] ------------------------------ Message: 2 Date: Sun, 05 Jan 2014 15:41:18 -0500 From: Duncan Murdoch <murdoch.duncan at gmail.com> To: Boris Aronshtam <baronshtam at lightminersystems.com>, "r-devel at r-project.org" <r-devel at r-project.org> Subject: Re: [Rd] Support for long arrays Message-ID: <52C9C36E.6030903 at gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed On 14-01-05 3:18 PM, Boris Aronshtam wrote:
My machine has hundreds of GB of RAM. Is there a way to create an array having more than 2 Billion (2^31) rows?
Not currently. Individual dimensions are limited to signed 32 bit values. The overall count of elements is limited to 2^52 or so. Duncan Murdoch
I am trying to create a large array using: d = c(1e+10,2) a=array(0,d) This results in the error: Error in array(0, d) : negative length vectors are not allowed In addition: Warning message: In array(0, d) : NAs introduced by coercion Note that I can create a vector of the same large size: v=c(1:2e+10) [[alternative HTML version deleted]]
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
------------------------------