[R-pkg-devel] Too many cores used in examples (not caused by data.table)
You are not the only one; I did the same with some of my examples. Would it be an option to ask for a default R-option, 'max.ncores', that specifies the maximum number of cores a process is allowed to use? CRAN could then require that that examples, tests and vignettes respect this option. That way there would be one uniform option to specify the maximum number of cores processes could use. That would also make it easier for system administrators to set default values for this (use the entire system; or use one code by default on a shared system). Of course, we package maintainers could do this without involvement of R-code or CRAN. We only need to agree on a name and a default value for when the option is missing (0 = use all cores; 1 or 2; or ncores-1 ...). Jan
On 24-10-2023 13:03, Greg Hunt wrote:
In my case recently, after an hour or so?s messing about I disabled some tests and example executions to get rid of the offending times. I doubt that i am the only one to do that. On Tue, 24 Oct 2023 at 9:38 pm, Helske, Jouni <jouni.helske at jyu.fi> wrote:
Thanks for the help, I now tried resubmitting with
Sys.setenv("OMP_THREAD_LIMIT" = 2) at the top of the exchange example, but
I still get the same note:
Examples with CPU time > 2.5 times elapsed time
user system elapsed ratio
exchange 1.196 0.04 0.159 7.774
Not sure what to try next.
Best,
Jouni
________________________________
From: Ivan Krylov <krylov.r00t at gmail.com>
Sent: Friday, October 20, 2023 16:54
To: Helske, Jouni <jouni.helske at jyu.fi>
Cc: r-package-devel at r-project.org <r-package-devel at r-project.org>
Subject: Re: [R-pkg-devel] Too many cores used in examples (not caused by
data.table)
? Thu, 19 Oct 2023 05:57:54 +0000
"Helske, Jouni" <jouni.helske at jyu.fi> ?????:
But I just realised that bssm uses Armadillo via RcppArmadillo, which
uses OpenMP by default for some elementwise operations. So, I wonder
if that could be the culprit?
I wasn't able to reproduce the NOTE either, despite manually setting
the environment variable
_R_CHECK_EXAMPLE_TIMING_CPU_TO_ELAPSED_THRESHOLD=2 before running R CMD
check, but I think I can see the code using OpenMP. Here's what I did:
0. Temporarily lower the system protections against capturing
performance traces of potentially sensitive parts:
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
(Set it back to 3 after you're done.)
1. Run the following command with the development version of the
package installed:
env OPENBLAS_NUM_THREADS=1 \
perf record --call-graph drawf,4096 \
R -e 'library(bssm); system.time(replicate(100, example(exchange)))'
OPENBLAS_NUM_THREADS=1 will prevent OpenBLAS from spawning worker
threads if you have it installed. (A different BLAS may need different
environment variables.)
2. Run `perf report` and browse collected call stack information.
The call stacks are hard to navigate, but I think they are not pointing
towards Armadillo. At least, setting ARMA_OPENMP_THREADS=1 doesn't
help, but setting OMP_THREAD_LIMIT=1 does.
--
Best regards,
Ivan
[[alternative HTML version deleted]]
______________________________________________
R-package-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel