Hi all, today, I stumbled upon a puzzling (to me) problem apparently related to do.call() that resulted in an efficiency drop of multiple orders of magnitudes compared to just calling the function directly (multiple minutes as compared to one second). That is fun(a = a, b = b, c = c, ...) took one second, while args <- list(a = a, b = b, c = c, ...) do.call(fun, args) took multiple minutes. In my package (brms), I use do.call in various places but only in one it resulted in this efficiency drop. Before I try to make a reproducible example, I wanted to ask if there are any known issues with do.call that may explain this? Paul
[R-pkg-devel] Effieciency drop in do.call?
5 messages · Gabor Grothendieck, Paul Buerkner, Peter Dalgaard
The do.call version evaluates all arguments while the normal version
may not depending on the function. There could also be a difference
if the function uses non-standard evaluation since in that case the
two could be passing different different argument values.
For an example of the second case,
f <- function(x) deparse(substitute(x))
f(pi)
## [1] "pi"
do.call("f", list(pi))
## [1] "3.14159265358979"
On Mon, Nov 19, 2018 at 11:50 AM Paul Buerkner <paul.buerkner at gmail.com> wrote:
Hi all,
today, I stumbled upon a puzzling (to me) problem apparently related to
do.call() that resulted
in an efficiency drop of multiple orders of magnitudes compared to just
calling the function directly (multiple minutes as compared to one second).
That is
fun(a = a, b = b, c = c, ...)
took one second, while
args <- list(a = a, b = b, c = c, ...)
do.call(fun, args)
took multiple minutes.
In my package (brms), I use do.call in various places but only in one it
resulted in this
efficiency drop.
Before I try to make a reproducible example, I wanted to ask if there are
any known issues
with do.call that may explain this?
Paul
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
If it was just about args evaluation, then the slowness would be in the list() call, no? An accidental deparse of a large structure could well be the culprit. -pd
On 19 Nov 2018, at 18:53 , Gabor Grothendieck <ggrothendieck at gmail.com> wrote:
The do.call version evaluates all arguments while the normal version
may not depending on the function. There could also be a difference
if the function uses non-standard evaluation since in that case the
two could be passing different different argument values.
For an example of the second case,
f <- function(x) deparse(substitute(x))
f(pi)
## [1] "pi"
do.call("f", list(pi))
## [1] "3.14159265358979"
On Mon, Nov 19, 2018 at 11:50 AM Paul Buerkner <paul.buerkner at gmail.com> wrote:
Hi all,
today, I stumbled upon a puzzling (to me) problem apparently related to
do.call() that resulted
in an efficiency drop of multiple orders of magnitudes compared to just
calling the function directly (multiple minutes as compared to one second).
That is
fun(a = a, b = b, c = c, ...)
took one second, while
args <- list(a = a, b = b, c = c, ...)
do.call(fun, args)
took multiple minutes.
In my package (brms), I use do.call in various places but only in one it
resulted in this
efficiency drop.
Before I try to make a reproducible example, I wanted to ask if there are
any known issues
with do.call that may explain this?
Paul
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
Thanks! You are likely right that this was the problem. Actually, I was using a pattern of the form args <- nlist(a, b, c, ...) do.call(fun, args) where nlist() uses NSE to set names for unnamed arguments. As I was passing big model objects and data through multiple layers of do.call(), it is not unlikely that some ended up being deparsed in the wrong way when evaluated in do.call(). Paul Am Mo., 19. Nov. 2018 um 22:33 Uhr schrieb peter dalgaard <pdalgd at gmail.com
:
If it was just about args evaluation, then the slowness would be in the list() call, no? An accidental deparse of a large structure could well be the culprit. -pd
On 19 Nov 2018, at 18:53 , Gabor Grothendieck <ggrothendieck at gmail.com>
wrote:
The do.call version evaluates all arguments while the normal version
may not depending on the function. There could also be a difference
if the function uses non-standard evaluation since in that case the
two could be passing different different argument values.
For an example of the second case,
f <- function(x) deparse(substitute(x))
f(pi)
## [1] "pi"
do.call("f", list(pi))
## [1] "3.14159265358979"
On Mon, Nov 19, 2018 at 11:50 AM Paul Buerkner <paul.buerkner at gmail.com>
wrote:
Hi all, today, I stumbled upon a puzzling (to me) problem apparently related to do.call() that resulted in an efficiency drop of multiple orders of magnitudes compared to just calling the function directly (multiple minutes as compared to one
second).
That is fun(a = a, b = b, c = c, ...) took one second, while args <- list(a = a, b = b, c = c, ...) do.call(fun, args) took multiple minutes. In my package (brms), I use do.call in various places but only in one it resulted in this efficiency drop. Before I try to make a reproducible example, I wanted to ask if there
are
any known issues
with do.call that may explain this?
Paul
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
A classical way of encountering this is
x <- rnorm(1000)
do.call("plot", list(x))
A way out is
do.call("plot", list(quote(x)))
-pd
On 19 Nov 2018, at 22:32 , peter dalgaard <pdalgd at gmail.com> wrote: If it was just about args evaluation, then the slowness would be in the list() call, no? An accidental deparse of a large structure could well be the culprit. -pd
On 19 Nov 2018, at 18:53 , Gabor Grothendieck <ggrothendieck at gmail.com> wrote:
The do.call version evaluates all arguments while the normal version
may not depending on the function. There could also be a difference
if the function uses non-standard evaluation since in that case the
two could be passing different different argument values.
For an example of the second case,
f <- function(x) deparse(substitute(x))
f(pi)
## [1] "pi"
do.call("f", list(pi))
## [1] "3.14159265358979"
On Mon, Nov 19, 2018 at 11:50 AM Paul Buerkner <paul.buerkner at gmail.com> wrote:
Hi all,
today, I stumbled upon a puzzling (to me) problem apparently related to
do.call() that resulted
in an efficiency drop of multiple orders of magnitudes compared to just
calling the function directly (multiple minutes as compared to one second).
That is
fun(a = a, b = b, c = c, ...)
took one second, while
args <- list(a = a, b = b, c = c, ...)
do.call(fun, args)
took multiple minutes.
In my package (brms), I use do.call in various places but only in one it
resulted in this
efficiency drop.
Before I try to make a reproducible example, I wanted to ask if there are
any known issues
with do.call that may explain this?
Paul
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com