Skip to content

[R-meta] Observed effect size using escalc

7 messages · Myers, Brett, Michael Dewey, Viechtbauer Wolfgang (STAT)

Myers, Brett
# 
Hello,



I am new to the metafor package and I am trying to decipher someone else's code. I'm specifically curious about the "yi" variable associated with the escalc() function. It appears that the yi vector is an output variable that escalc() appends to the dataset. The metafor documentation says that yi is the observed effect size, and we didn't enter this as an input variable, so I'm wondering what formula escalc() uses to determine yi. Would anyone here have some insight on how yi is calculated?



Thanks,

Brett Myers
Myers, Brett
# 
Hello,

I am new to the metafor package and I am trying to decipher someone else's code. I'm specifically curious about the "yi" variable associated with the escalc() function. It appears that the yi vector is an output variable that escalc() appends to the dataset. The metafor documentation says that yi is the observed effect size, and we didn't enter this as an input variable, so I'm wondering what formula escalc() uses to determine yi. Would anyone here have some insight on how yi is calculated?

Thanks,
Brett Myers
Michael Dewey
# 
Dear Brett

It calculates it using one of dozens of formulae depending on what sort 
of input you gave it. If you show us the code you have inherited we can 
probably help further. The help ?escalc does have wagon-loads of detail 
about this if you want to probe further.

When you do post code please set your e-mailer to send plain text as 
HTML gets mangled in transmission and can make you message unreadable.
On 11/08/2017 16:21, Myers, Brett wrote:

  
    
  


  


      

    

    

      
      

        


  

        

      Myers, Brett
    

    

      
      #
    

  


  

      
Michael,

Thank you for your reply. Here is the pertinent line of code from which yi is derived:

meta_escalc <- escalc(measure="SMD", n1i=n, n2i=n.1, m1i=(post_mean-pre_mean), m2i=(post_mean.1-pre_mean.1), sd1i=pre_sd, sd2i=pre_sd.1, data=newdata)

In the summary data output, yi is introduced, so I assume escalc is calculating the effect size based on our input variables. I was under the impression effect size would be calculated as:
ES = [(post_mean-pre_mean) - (post_mean.1-pre_mean.1)] / (pooled pretest sd)

However, the solution of ES by hand does not match the yi output from escalc, which leads me to wonder what the calculation actually is. Any thoughts on this?

Brett

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Viechtbauer Wolfgang (STAT)
    

    

      
      #
    

  


  

      
For measure="SMD", escalc() computes:

ni <- n1i + n2i
mi <- ni - 2
sdpi <- sqrt(((n1i-1)*sd1i^2 + (n2i-1)*sd2i^2)/mi)
di <- (m1i - m2i) / sdpi
cmi <- .cmicalc(mi)
yi <- cmi * di

where .cmicalc() is a function that computes the bias correction factor, which is given by

exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2)

The reference for this is Hedges (1981).

Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.

But, based on the naming of the variables, it is pretty clear that the escalc() function is being misused here. For m1i and m2i, mean changes are provided but pre (treatment?) SDs for sd1i and sd2i. This invalidates the assumption underlying the computations.

If you are dealing with two-group pre-post designs, then you will want to use measures like "SMCC" or "SMCR" (computed for each group separately) and then taking the difference as the effect size. See also:

http://www.metafor-project.org/doku.php/analyses:morris2008

Best,
Wolfgang

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Myers, Brett
    

    

      
      #
    

  


  

      
Thank you so much for that clarification! I really appreciate it.

Brett

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Viechtbauer Wolfgang (STAT)
    

    

      
      #
    

  


  

      
Forgot the closing parentheses on the equation for the bias correction factor. It should be:

exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2))

Also, usually the equation is written as:

gamma(mi/2) / (sqrt(mi/2) * gamma((mi-1)/2))

but this can lead to numerical overflow when mi is large. An example:

mi <- 500
gamma(mi/2) / (sqrt(mi/2) * gamma((mi-1)/2))
exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2))

Often, the following approximation is used:

1 - 3 / (4*mi - 1)

which works very well, even when mi is small.

Best,
Wolfgang

-----Original Message-----
From: Myers, Brett [mailto:brett.myers at vanderbilt.edu] 
Sent: Friday, August 11, 2017 23:52
To: Viechtbauer Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: RE: [R-meta] Observed effect size using escalc

Thank you so much for that clarification! I really appreciate it.

Brett