Message-ID: <BANLkTimZTK2uEFmXosVE0SUhTF8ALWsawQ@mail.gmail.com>
Date: 2011-04-13T18:11:25Z
From: Jay Emerson
Subject: [Rcpp-devel] Use size_t rather than int for vector sizes and indexing?
In-Reply-To: <7f7a8d3086cc4d8bffb93df23ea2c3de@surf-chuck.com>
Chuck,
Internally, R is using 4-byte integers for indexing, and the length of
a vector is thus constrained to 2-billion-ish elements. Two ways
around this include packages ff and
bigmemory, for example, or relying on database-like queries. However,
the resulting objects can't be used with standard R functions (with
some special exceptions).
Jay
On Wed, Apr 13, 2011 at 2:05 PM, <chuck at chuckcode.com> wrote:
>
> Hi All,
>
> Thanks to Dirk Eddelbuettel and the other contributors for such a
> wonderful package. Rcpp really transforms the way that I've been able to
> incorporate c++ code into R. Makes it possible to speed up the critical
> computations while keeping all the great flexibility and features of R.
>
> I've run into a problem lately with particularly large vectors in Rcpp. I
> seem to be overflowing when my vectors get larger than 2^31 elements on a
> 64 bit system. It looks from the code of both the classic (included below)
> and the newer versions as though this is due to using ints rather than
> something like size_t (http://en.wikipedia.org/wiki/Size_t) as the type for
> size and indexing into the vector. It looks like RcppResultsSet's add()
> functions may also be using ints.
>
> Curious to know if there is a particular reason for using ints rather than
> something like size_t and if the project managers would be open to changing
> to accomodate larger vectors.
>
> Thanks,
> -Chuck Sugnet
>
> template <typename T>
> class RcppVector {
> public:
> ? ? ? ?typedef T* iterator ;
> ? ? ? ?typedef const T* const_iterator ;
>
> ? ?RcppVector(SEXP vec);
> ? ?RcppVector(int len);
> ? ?int size() const;
> ? ?T& operator()(int i) const;
> ? ?T *cVector() const;
> ? ?std::vector<T> stlVector() const;
>
> ? ?inline const_iterator begin() const { return v ; }
> ? ?inline const_iterator end() const { return v + len ; }
>
> ? ?inline iterator begin(){ return v ; }
> ? ?inline iterator end(){ return v + len ; }
>
> private:
> ? ?int len;
> ? ?T *v;
> };
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-devel at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
>
--
John W. Emerson (Jay)
Associate Professor of Statistics
Department of Statistics
Yale University
http://www.stat.yale.edu/~jay