Message-ID: <CAPr7RtUd99n_VeYG0AFnBe5ZfgprgZWuW4qgdNSLvgS+9guJxQ@mail.gmail.com>
Date: 2020-07-08T04:20:39Z
From: ivo welch
Subject: parallel number of cores according to memory?
if I understand correctly, R makes a copy of the full environment for each
process. thus, even if I have 32 processors, if I only have 64GB of RAM
and my R process holds about 10GB, I should probably not spawn 32 processes.
has anyone written a function that sets the number of cores for use (in
mclapply) to be guessed at by appropriate memory requirements (e.g.,
"amount-of-RAM"/"RAM held by R")?
(it would be even nicer if I could declare my 8GB data frame to be
read-only and to be shared among my processes, but this is presumably
technically very difficult.)
pointers appreciated.
/iaw
[[alternative HTML version deleted]]