Skip to content

summary() dispatch puzzle

3 messages · Martin Maechler, Ben Bolker

Original
Ben Bolker
# 
I'm sorry I haven't boiled this down to a more-minimal example yet,
but ...

  I'm working on an S3 method (tidy.merMod, in the 'broom' package). It
normally handles 'merMod' objects from the lme4 package, but I'm trying
to make it handle 'merModLmerTest' objects from the lmerTest package too.

  The merModLmerTest class inherits (as an S4) class from the merMod
class.  The important difference (for my current purposes) is that
summary.merMod returns an object containing a coefficient table
*without* degrees or freedom or p-values, while the summary method for
merModLmerTest objects returns one *with* both of those columns.

 Because merModLmerTest inherits from merMod, calling tidy(obj) on a
merModLmerTest object does get you into broom:::tidy.merMod.  Within
that function, though, calling summary() appears to call summary.merMod
instead of merModLmerTest, so I don't get the additional columns I want.

  Ideas/directions for further diagnoses?  Is this an S3/S4 confusion,
or something funny about the way the broom and merModLmerTest package
are talking to each other, or ... ?

  cheers
    Ben Bolker


----
library(lme4)
fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
library(lmerTest)
fm1B <- as(fm1,"merModLmerTest")
coef(summary(fm1B))               ## table with df and p values
coef(lme4:::summary.merMod(fm1B)) ## table without df and p values

is(fm1B,"merMod")  ## TRUE

library(broom)
tidy(fm1B)
## hard to show what's going on without debugging through the function:
## the issue occurs here:
## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94

I can't replicate this behaviour with a trivial method ...

silly <- function(x, ...) {
    UseMethod("silly")
}
silly.merMod <- function(object, ...) {
    coef(summary(object))
}
silly(fm1B)
environment(silly) <- environment(tidy)
silly(fm1B)
2 days later
Martin Maechler
# 
> I'm sorry I haven't boiled this down to a more-minimal example yet,
    > but ...

    > I'm working on an S3 method (tidy.merMod, in the 'broom' package). It
    > normally handles 'merMod' objects from the lme4 package, but I'm trying
    > to make it handle 'merModLmerTest' objects from the lmerTest package too.

    > The merModLmerTest class inherits (as an S4) class from the merMod
    > class.  The important difference (for my current purposes) is that
    > summary.merMod returns an object containing a coefficient table
    > *without* degrees or freedom or p-values, while the summary method for
    > merModLmerTest objects returns one *with* both of those columns.

    > Because merModLmerTest inherits from merMod, calling tidy(obj) on a
    > merModLmerTest object does get you into broom:::tidy.merMod.  Within
    > that function, though, calling summary() appears to call summary.merMod
    > instead of merModLmerTest, so I don't get the additional columns I want.

    > Ideas/directions for further diagnoses?  Is this an S3/S4 confusion,
    > or something funny about the way the broom and merModLmerTest package
    > are talking to each other, or ... ?

I'm not sure, if I see your problem when I execute the
reproducible code below ... 


    > ----
    > library(lme4)
    > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
    > library(lmerTest)
    > fm1B <- as(fm1,"merModLmerTest")
    > coef(summary(fm1B))               ## table with df and p values
    > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values

    > is(fm1B,"merMod")  ## TRUE

    > library(broom)
    > tidy(fm1B)

Part of what I see from above, is

    > coef(summary(fm1B))               ## table with df and p values
		 Estimate Std. Error       df   t value     Pr(>|t|)
    (Intercept) 251.40510   6.824556 17.00002 36.838310 0.000000e+00
    Days         10.46729   1.545789 17.00000  6.771485 3.263788e-06

    > tidy(fm1B)
			      term     estimate std.error statistic    group
    1                  (Intercept) 251.40510485  6.824556 36.838310    fixed
    2                         Days  10.46728596  1.545789  6.771485    fixed
    3       sd_(Intercept).Subject  24.74044768        NA        NA  Subject
    4              sd_Days.Subject   5.92213326        NA        NA  Subject
    5 cor_(Intercept).Days.Subject   0.06555134        NA        NA  Subject
    6      sd_Observation.Residual  25.59181589        NA        NA Residual
    > 

which seems consistent at least: You get std.error and statistic
for those two terms that you get 'df' and 'p-value from "the
good method" of summary(.)

?

Of course, I'm using the CRAN versions of the packages, not
github ones, and I see that we end up using half of tidyVerse
... which does not necessarily keep debugging easy..

Martin :
R version 3.3.1 Patched (2016-07-16 r70928)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Fedora 22 (Twenty Two)

locale:
 [1] LC_CTYPE=de_CH.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=de_CH.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=de_CH.UTF-8   
 [7] LC_PAPER=de_CH.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_CH.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] graphics  grDevices datasets  stats     utils     methods   base     

other attached packages:
[1] broom_0.4.1     lmerTest_2.0-32 lme4_1.1-12     Matrix_1.2-3    fortunes_1.5-3 
[6] sfsmisc_1.1-0  

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.5         compiler_3.3.1      nloptr_1.0.4        RColorBrewer_1.1-2 
 [5] plyr_1.8.4          tools_3.3.1         rpart_4.1-10        tibble_1.1         
 [9] gtable_0.2.0        nlme_3.1-128        lattice_0.20-33     psych_1.6.6        
[13] DBI_0.4-1           parallel_3.3.1      gridExtra_2.2.1     stringr_1.0.0      
[17] dplyr_0.5.0         cluster_2.0.3       grid_3.3.1          nnet_7.3-12        
[21] data.table_1.9.6    R6_2.1.2            survival_2.39-5     foreign_0.8-66     
[25] latticeExtra_0.6-28 minqa_1.2.4         Formula_1.2-1       tidyr_0.5.1        
[29] reshape2_1.4.1      ggplot2_2.1.0       magrittr_1.5        Hmisc_3.17-4       
[33] scales_0.4.0        MASS_7.3-45         splines_3.3.1       assertthat_0.1     
[37] mnormt_1.5-4        colorspace_1.2-6    stringi_1.1.1       acepack_1.3-3.3    
[41] munsell_0.4.3       chron_2.3-47
> ## hard to show what's going on without debugging through the function:
    > ## the issue occurs here:
    > ## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94

    > I can't replicate this behaviour with a trivial method ...

    > silly <- function(x, ...) {
    > UseMethod("silly")
    > }
    > silly.merMod <- function(object, ...) {
    > coef(summary(object))
    > }
    > silly(fm1B)
    > environment(silly) <- environment(tidy)
    > silly(fm1B)

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel
Ben Bolker
# 
I'm not doing a good job at explaining.  See inline below ...
On 16-07-18 11:02 AM, Martin Maechler wrote:
This is as expected.  What's going on here is that I'm trying to get
the p-value into the tidied object.   You really have to debug *into*
the function to see the problem.  I've tried to remedy this a bit in my
fork by adding a 'debug' argument:

devtools::install_github("bbolker/broom")
## ... re-load package or re-run session ...
tidy(fm1B,debug=TRUE)

produces

output from coef(summary(x)):
             Estimate Std. Error   t value
(Intercept) 251.40510   6.824556 36.838310
Days         10.46729   1.545789  6.771485


that is, when coef(summary(x)) is run *inside* the tidy.merMod function,
it calls summary.merMod rather than the summary method for
merModLmerTest objects ...
Sorry about that.  broom is fairly heavily integrated with the
tidyverse ...