lm Regression takes 24+ GB RAM - Error message

Le mercredi 06 mars 2013 ? 08:31 -0800, Jonas125 a ?crit :
The datatable (and the split obviously) only contain characters and numeric
data.

I found that 4 regression in a row work if I don't use the calculated
columns as variables but 2 of the original columns. 
RAM usage stays below 3GB!
--> Why does R has such problems with the calculated columns? Their
calculation is already done before the regression starts. 

It's like this:
Create the calculated columns:
Dataset$ExtraColumn1 <- Dataset$ColumnA / Dataset$ColumnB
Dataset$ExtraColumn2 <- Dataset$ColumnC / Dataset$ColumnD

Perform the split of the dataset inc. calculated columns (the criteria for
the split have a hierarchy):
Datasplit <- split(Dataset, paste(Dataset$ColumnE, Dataset$ColumnE))

Perform the regression on the splitted data:
Regression1 <- lapply(Datasplit, function(d) lm(ExtraColumn1 ~ ExtraColumn2,
d, na.action = na.omit, singular.ok = TRUE))

BTW: There are no NA values in the data source.

What is my mistake?
What's the value of length(Datasplit)? Have you tried running
regressions manually on Datasplit[[1]] and calling object.size() on the
result to see how large it is?

Regards
When I calculate the columns I might divide by zero (=inf). Could that
create the problem in the regression?

Thanks,
Jonas

--
View this message in context: http://r.789695.n4.nabble.com/lm-Regression-takes-24-GB-RAM-Error-message-tp4660434p4660496.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

lm Regression takes 24+ GB RAM - Error message

Thread (6 messages)