[R-pkg-devel] Replacement for SETLENGTH
On Wed, 15 Jan 2025 15:30:36 +0000
"Merlise Clyde, Ph.D." <clyde at duke.edu> wrote:
For memory limited machines, the alloc/copy was a problem for memory usage - and if I recall was one of the reasons in 2008 I switched to SETLENGTH, which doesn't seem to do an allocation ??? If there is going to be an absolute ban on SETLENGTH in packages I'll probably need to address memory management differently for those cases.
If you need to adjust the xlength() of your vectors without causing reallocations, I'm afraid the only API-compliant way to do that for now is ALTREP [1]. It's a lot of typing because the code will need to register an ALTREP class for every vector type that needs to be shrinkable. Moreover, since some of those vectors are of type VECSXP and "altlist" classes only appeared in R-4.3.0, you will probably prefer to leave the old implementation based on SETLENGTH() behind #if R_VERSION < R_Version(4, 3, 0) to avoid requiring your users to upgrade from R >= 3.0. A work-in-progress implementation of shrinkable vectors for data.table can be found at [2]. The real problem is not implementing the same simple API using SETLENGTH() and ALTREP, but in refactoring the rest of the code not to violate its assumptions. (Total control over the allocation of your shrinkable vectors is required: nothing good will happen if the code tries to call ALTREP methods on an object that is not of the exact ALTREP class it needs to be.) My own notes on the use of ALTREP can be found at [3]. I will do my best to keep them correct, but patches are always welcome. And now a question: Would R benefit from a patch to make xlengthgets() more like EnlargeVector() [4] and sometimes return vectors with GROWABLE_BIT set? (EnlargeVector() is currently only reachable from the `[<-` operation.) It wouldn't be a deviation from the currently documented behaviour if the function tested for NO_REFERENCES() and then used SETLENGTH() when, say, reducing the length of a vector to 1/2 of its length or more.
Best regards, Ivan [1] https://rdatatable-community.github.io/The-Raft/posts/2025-01-13-non-api-use/index.html#growable-vectors [2] https://github.com/Rdatatable/data.table/blob/growable_refactor/src/growable.c Everyone is welcome to use this under the terms of GPL-2 or a later version, or MPL-2, at their choice. [3] https://aitap.codeberg.page/R-api/#ALTREP [4] https://github.com/r-devel/r-svn/blob/59df414b844eea27c09b352ad30fd82e66764d2d/src/main/subassign.c#L256-L260