Moderating consequences of garbage collection when in C
On 11/14/2011 01:12 PM, dhinds at sonic.net wrote:
Martin Morgan<mtmorgan at fhcrc.org> wrote:
On 11/14/2011 11:47 AM,dhinds at sonic.net wrote:
> dhinds at sonic.net wrote:
>> Martin Morgan<mtmorgan at fhcrc.org> wrote:
> > I had done some google searches on this issue, since it seemed like it > should not be too uncommon, but the only other hit I could come up > with was a thread from 2006: > > https://stat.ethz.ch/pipermail/r-devel/2006-November/043446.html > > In any case, one issue with your suggested workaround is that it > requires knowing how much additional storage is needed, which may be > an expensive operation to determine. I've just tried implementing a > different approach, which is to define two new functions to either > disable or enable GC. The function to disable GC first invokes > R_gc_full() to shrink the heap as much as possible, then sets a flag. > Then in R_gc_internal(), I first check that flag, and if it is set, I > call AdjustHeapSize(size_needed) and exit immediately.
I think this is a better approach; mine seriously understated the complexity of figuring out required size.
> These calls could be used to bracket any code section that expects to > make lots of calls to R's memory allocator. The down side is that > this approach requires that all paths out of such a code section > (including error handling) need to take care to unset the GC-disabled > flag. I think I would want to hear from someone on the R team about > whether they think this is a good idea. >
Another place where this comes up is during package load, especially for packages with many S4 instances.
Do you know if this is all happening inside a C function that could handle disabling and enabling GC? Or would it require doing this at the R level? For testing, I am turning GC on and off at the R level
Generally complicated operations across multiple function calls.
Something like
f = function() {
state <- gcdisable(TRUE)
on.exit(gcdisable(state))
as.character(1:10000000)
}
might be used.
Martin
Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793