On 13.08.2023 08:14, Ivan Krylov wrote:
? Sat, 12 Aug 2023 22:49:01 -0700
Michael Topper <miketopper123 at gmail.com> ?????:
It appears that some of my examples/tests are taking too
long to run for CRAN's standards.
I don't think they are running too long; I think they are too parallel.
The elapsed time is below 1s, but the "user" time (CPU time spent in
the process) is 7 to 13 times that. This suggests that your code
resulted in starting more threads than CRAN allows (up to 2 if you have
to test parallellism). Are you using OpenMP? data.table? makeCluster()?
It's simplest to always to default to a parallelism factor of 2 in
examples an tests, because determining the right number is a hard
problem. (What if the computer is busy doing something else? What if
the BLAS is already parallel enough?)
Moreover, is there any insight as to why this would happen on the
third update of the package rather than on the first or second?
The rule has always depended on the particular system running the
checks (five seconds on my 12-year-old ThinkPad or on my ultraportative
with an Intel Atom that had snails in its ancestry?). Maybe some
dependency of your package has updated and started creating threads
where it previously didn't.
Good points, not only for examples and tests, but also for defaults.
On shared resources (such as clusters) users may not expect the
parallelization you use and then overallocate the resources.
Example: 20 cores available to the user who runs makeCluster() for
paallelization, but the underlying code does multihreading on 20 cores.
Then we end up in 20*20 threads on the machine slowing down the machine
and processes of other uers.
Hence, defaults should also not be more than 2. Simply allow the user to
ask for more.
Best,
Uwe Ligges