Dear all, I am trying to benchmark code that occasionally prints on the screen and I want to suppress the printing. Is there an idiom for this? If I do sink(tempfile) microbenchmark(...) sink() then I'll be also measuring the costs of writing to tempfile. I could also sink to /dev/null, which is probably fast, but that is not portable. Is there a better solution? Is writing to a textConnection() better? Thanks, Best, Gabor
Benchmark code, but avoid printing
4 messages · Gábor Csárdi, Henrik Bengtsson, Simon Urbanek
On Fri, Jan 2, 2015 at 9:02 AM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote:
Dear all, I am trying to benchmark code that occasionally prints on the screen and I want to suppress the printing. Is there an idiom for this? If I do sink(tempfile) microbenchmark(...) sink() then I'll be also measuring the costs of writing to tempfile. I could also sink to /dev/null, which is probably fast, but that is not portable.
Interesting problem. On Windows NUL corresponds to /dev/NULL, e.g.
con <- file("NUL", open="wb"). Not that it's cross platform, but it
at least allows you to cover on more OS. Maybe R should have a
built-in "null" device. An easier solution is probably to go back to
the maintainers of the functions outputting text and ask them for an
option to disable that.
Is there a better solution? Is writing to a textConnection() better?
For large number of output *lines* (not characters), textConnection() is exponentially slow (at least in R 3.1.0). Use rawConnection() instead, cf. http://www.jottr.org/2014/05/captureOutput.html /Henrik
Thanks, Best, Gabor
______________________________________________ R-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
On Jan 2, 2015, at 12:02 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote:
Dear all, I am trying to benchmark code that occasionally prints on the screen and I want to suppress the printing. Is there an idiom for this? If I do sink(tempfile) microbenchmark(...) sink() then I'll be also measuring the costs of writing to tempfile. I could also sink to /dev/null, which is probably fast, but that is not portable. Is there a better solution? Is writing to a textConnection() better?
Define better - you're just trading off one output code for another - it will be still measuring the cost of the output, obviously, and since the output is part of the code you're profiling it's correctly so. Each output method has different beavior - e.g. text connection can be faster, but it can also trigger additional garbage collection so it will affect results. Example:
f=textConnection("x", "w")
sink(f)
m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
sink()
m
Unit: milliseconds
expr
{ for (i in 1:100) { print("foo") sum(rnorm(1000)) } }
min lq mean median uq max neval
12.76462 15.34483 17.85341 17.02435 19.56384 63.09329 100
sink("/dev/null")
m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
sink()
m
Unit: milliseconds
expr
{ for (i in 1:100) { print("foo") sum(rnorm(1000)) } }
min lq mean median uq max neval
13.0191 13.03601 13.41815 13.0534 13.16496 16.25288 100
As you can see /dev/null is more predictable, because it's straight output, but text connection can be faster in the beginning and becomes progressively slower.
As Henrik said, you're probably best off using simply /dev/null - the only oddball is Windows, and that's a trivial condition on .Platform$OS.type.
Cheers,
S
Yes, thanks much, this makes a lot of sense. Well, by "better" what I had in mind was something that is reliably close to the time needed for printing. Without actually doing the printing. But I realize this is too much to ask for, and I'll be fine with /dev/null. Thanks for bringing up the textConnection() issue as well, especially because I am using textConnection now. /dev/null is a better option. Best, Gabor On Fri, Jan 2, 2015 at 3:44 PM, Simon Urbanek
<simon.urbanek at r-project.org> wrote:
On Jan 2, 2015, at 12:02 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote:
Dear all, I am trying to benchmark code that occasionally prints on the screen and I want to suppress the printing. Is there an idiom for this? If I do sink(tempfile) microbenchmark(...) sink() then I'll be also measuring the costs of writing to tempfile. I could also sink to /dev/null, which is probably fast, but that is not portable. Is there a better solution? Is writing to a textConnection() better?
Define better - you're just trading off one output code for another - it will be still measuring the cost of the output, obviously, and since the output is part of the code you're profiling it's correctly so. Each output method has different beavior - e.g. text connection can be faster, but it can also trigger additional garbage collection so it will affect results. Example:
f=textConnection("x", "w")
sink(f)
m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
sink()
m
Unit: milliseconds
expr
{ for (i in 1:100) { print("foo") sum(rnorm(1000)) } }
min lq mean median uq max neval
12.76462 15.34483 17.85341 17.02435 19.56384 63.09329 100
sink("/dev/null")
m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
sink()
m
Unit: milliseconds
expr
{ for (i in 1:100) { print("foo") sum(rnorm(1000)) } }
min lq mean median uq max neval
13.0191 13.03601 13.41815 13.0534 13.16496 16.25288 100
As you can see /dev/null is more predictable, because it's straight output, but text connection can be faster in the beginning and becomes progressively slower.
As Henrik said, you're probably best off using simply /dev/null - the only oddball is Windows, and that's a trivial condition on .Platform$OS.type.
Cheers,
S