Le mercredi 06 mars 2013 ? 09:18 -0800, Jonas125 a ?crit :
Length(Datasplit) = 7100
I did a regression for Datasplit[[1]] and the calculated columns --> the
object size is 70 MB. Quite large....
7100*70/1024 = 485 (GB)
No wonder why you run out of memory quite fast.
You probably do not need to store the whole lm objects: usually you need
coefficients, R-squared, things like that. So instead of returning the
objects, return a vector or a list with only the elements you need, you
will save much space.
And if you really need the objects, set these lm() arguments to FALSE to
make the result smaller:
model, x, y, qr: logicals. If ?TRUE? the corresponding components of
the fit (the model frame, the model matrix, the response, the
QR decomposition) are returned.
Assuming that R cannot handle inf values in regressions (didn't have the
time to google it)
How can I avoid the calculation of infinite values? Like "If the denominator
would be zero, choose 0.0000001 as the denominator instead."
Dataset[is.infinite(Dataset)] <- 0 does not work for me --> "default method
not implemented for type 'list' "
class(Dataset) = data.frame
I don't understand why you think infinite values can trigger a memory
problem. Why don't you just try it?
lm(c(1, Inf) ~ c(1, 2))
Erreur dans lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
NA/NaN/Inf in 'y'
lm(c(1, 2) ~ c(1, Inf))
Erreur dans lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
NA/NaN/Inf in 'x'
So, if anything, this would stop your lapply() call sooner or later, and
save your machine from freezing.
Regards