if I understand correctly, R makes a copy of the full environment for each
process. thus, even if I have 32 processors, if I only have 64GB of RAM
and my R process holds about 10GB, I should probably not spawn 32 processes.
has anyone written a function that sets the number of cores for use (in
mclapply) to be guessed at by appropriate memory requirements (e.g.,
"amount-of-RAM"/"RAM held by R")?
(it would be even nicer if I could declare my 8GB data frame to be
read-only and to be shared among my processes, but this is presumably
technically very difficult.)
pointers appreciated.
/iaw