summary() dispatch puzzle

I'm not doing a good job at explaining.  See inline below ...
Ben Bolker <bbolker at gmail.com>
    on Fri, 15 Jul 2016 16:45:50 -0400 writes:akram

    > I'm sorry I haven't boiled this down to a more-minimal example yet,
    > but ...

    > I'm working on an S3 method (tidy.merMod, in the 'broom' package). It
    > normally handles 'merMod' objects from the lme4 package, but I'm trying
    > to make it handle 'merModLmerTest' objects from the lmerTest package too.

    > The merModLmerTest class inherits (as an S4) class from the merMod
    > class.  The important difference (for my current purposes) is that
    > summary.merMod returns an object containing a coefficient table
    > *without* degrees or freedom or p-values, while the summary method for
    > merModLmerTest objects returns one *with* both of those columns.

    > Because merModLmerTest inherits from merMod, calling tidy(obj) on a
    > merModLmerTest object does get you into broom:::tidy.merMod.  Within
    > that function, though, calling summary() appears to call summary.merMod
    > instead of merModLmerTest, so I don't get the additional columns I want.

    > Ideas/directions for further diagnoses?  Is this an S3/S4 confusion,
    > or something funny about the way the broom and merModLmerTest package
    > are talking to each other, or ... ?
I'm not sure, if I see your problem when I execute the
reproducible code below ... 

    > ----
    > library(lme4)
    > fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
    > library(lmerTest)
    > fm1B <- as(fm1,"merModLmerTest")
    > coef(summary(fm1B))               ## table with df and p values
    > coef(lme4:::summary.merMod(fm1B)) ## table without df and p values

    > is(fm1B,"merMod")  ## TRUE

    > library(broom)
    > tidy(fm1B)
Part of what I see from above, is

    > coef(summary(fm1B))               ## table with df and p values
		 Estimate Std. Error       df   t value     Pr(>|t|)
    (Intercept) 251.40510   6.824556 17.00002 36.838310 0.000000e+00
    Days         10.46729   1.545789 17.00000  6.771485 3.263788e-06

    > tidy(fm1B)
			      term     estimate std.error statistic    group
    1                  (Intercept) 251.40510485  6.824556 36.838310    fixed
    2                         Days  10.46728596  1.545789  6.771485    fixed
    3       sd_(Intercept).Subject  24.74044768        NA        NA  Subject
    4              sd_Days.Subject   5.92213326        NA        NA  Subject
    5 cor_(Intercept).Days.Subject   0.06555134        NA        NA  Subject
    6      sd_Observation.Residual  25.59181589        NA        NA Residual
    > 
which seems consistent at least: You get std.error and statistic
for those two terms that you get 'df' and 'p-value from "the
good method" of summary(.)
This is as expected.  What's going on here is that I'm trying to get
the p-value into the tidied object.   You really have to debug *into*
the function to see the problem.  I've tried to remedy this a bit in my
fork by adding a 'debug' argument:

devtools::install_github("bbolker/broom")
## ... re-load package or re-run session ...
tidy(fm1B,debug=TRUE)

produces

output from coef(summary(x)):
             Estimate Std. Error   t value
(Intercept) 251.40510   6.824556 36.838310
Days         10.46729   1.545789  6.771485

that is, when coef(summary(x)) is run *inside* the tidy.merMod function,
it calls summary.merMod rather than the summary method for
merModLmerTest objects ...
?

Of course, I'm using the CRAN versions of the packages, not
github ones, and I see that we end up using half of tidyVerse
... which does not necessarily keep debugging easy..

Sorry about that.  broom is fairly heavily integrated with the
tidyverse ...
Martin :

sessionInfo()
R version 3.3.1 Patched (2016-07-16 r70928)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Fedora 22 (Twenty Two)

locale:
 [1] LC_CTYPE=de_CH.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=de_CH.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=de_CH.UTF-8   
 [7] LC_PAPER=de_CH.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_CH.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] graphics  grDevices datasets  stats     utils     methods   base     

other attached packages:
[1] broom_0.4.1     lmerTest_2.0-32 lme4_1.1-12     Matrix_1.2-3    fortunes_1.5-3 
[6] sfsmisc_1.1-0  

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.5         compiler_3.3.1      nloptr_1.0.4        RColorBrewer_1.1-2 
 [5] plyr_1.8.4          tools_3.3.1         rpart_4.1-10        tibble_1.1         
 [9] gtable_0.2.0        nlme_3.1-128        lattice_0.20-33     psych_1.6.6        
[13] DBI_0.4-1           parallel_3.3.1      gridExtra_2.2.1     stringr_1.0.0      
[17] dplyr_0.5.0         cluster_2.0.3       grid_3.3.1          nnet_7.3-12        
[21] data.table_1.9.6    R6_2.1.2            survival_2.39-5     foreign_0.8-66     
[25] latticeExtra_0.6-28 minqa_1.2.4         Formula_1.2-1       tidyr_0.5.1        
[29] reshape2_1.4.1      ggplot2_2.1.0       magrittr_1.5        Hmisc_3.17-4       
[33] scales_0.4.0        MASS_7.3-45         splines_3.3.1       assertthat_0.1     
[37] mnormt_1.5-4        colorspace_1.2-6    stringi_1.1.1       acepack_1.3-3.3    
[41] munsell_0.4.3       chron_2.3-47       

    > ## hard to show what's going on without debugging through the function:
    > ## the issue occurs here:
    > ## https://github.com/dgrtwo/broom/blob/master/R/lme4_tidiers.R#L94

    > I can't replicate this behaviour with a trivial method ...

    > silly <- function(x, ...) {
    > UseMethod("silly")
    > }
    > silly.merMod <- function(object, ...) {
    > coef(summary(object))
    > }
    > silly(fm1B)
    > environment(silly) <- environment(tidy)
    > silly(fm1B)

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

summary() dispatch puzzle

Thread (3 messages)