Skip to content

[R-meta] Capturing the variability at the lowest level of longitudinal studies

3 messages · Michael Dewey, Simon Harmel, Wolfgang Viechtbauer

Original
Michael Dewey
# 
Dear Simon

I am not the world's greatest expert on these designs but I would have 
included ~1 | esid not ~time | esid

Incidentally you posted in HTML not plain text which leads to messages 
getting scrmbled.

Michael
On 01/05/2021 00:41, Simon Harmel wrote:

  
    
  


  


    

    
1 day later

    

      
      

        


  

        

      Simon Harmel
    

    

      
      #
    

  


  

      
Dear All,

I have fit the model below. However, rho (= ICC) is estimated to be
"1.0000". Does this mean there is extreme variability between true effects
within each time point? (Or extreme dependence between true effects within
each time point?)

Should I try to find time-related moderators that can potentially explain
some of this variability in the true effects? How low an rho (=ICC) is
reasonably acceptable?

ps: if we set `struct= c("UN","UN")`, we get the following (is this
normal?):
    rho.1   rho.2   rho.3   rho.4         1   2   3   4
1       1  1.0000  1.0000  0.5000     -  no  no  no
2  1.0000       1  1.0000  0.5000    31   -  no  no
3  1.0000  1.0000       1  0.5000     3   3   -  no
4  0.5000  0.5000  0.5000       1     1   1   1   -

#-------- My current model:

rma.mv(yi ~ 0 + time + x1 + x2, V = V, struct= c("AR","UN"),
                    random = list(~time|study, ~1|esid),
                    data = data)

OUTPUT:
Variance Components:

                    estim    sqrt  nlvls  fixed  factor
sigma^2    0.2117  0.4601    255     no    esid

outer factor: study (nlvls = 49)
inner factor: time (nlvls = 4)

                estim    sqrt     fixed
tau^2      0.2679  0.5176     no
rho          1.0000                 no <<<--------- Isn't this odd?

Test for Residual Heterogeneity:
QE(df = 249) = 1432.5141, p-val < .0001

Test of Moderators (coefficients 1:6):
QM(df = 6) = 46.1983, p-val < .0001

Model Results:

          estimate         se        zval      pval        ci.lb     ci.ub
time1    0.5928  0.1221   4.8561  <.0001   0.3535  0.8320  ***
time2    0.5671  0.1346   4.2119  <.0001   0.3032  0.8310  ***
time3    0.6288  0.2328   2.7013  0.0069   0.1726  1.0850   **
time4    1.4178  0.4640   3.0556  0.0022   0.5084  2.3272   **
x1        -0.0114  0.0068  -1.6810  0.0928  -0.0248  0.0019    .
x2         0.0133  0.0207   0.6433  0.5200  -0.0272  0.0538

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
Dear Simon,

The rho values are correlations, so this has nothing to do with variability. High values of rho indicate strong correlations.

But in the model with "UN", three of those correlations are being estimated based on 1 pair, two are based on 3 pairs, and only the 1-2 pair occurs more often. So essentially, for most of those correlations, there is very little information to estimate them. Also, the corresponding variance components might be essentially 0, in which case one wouldn't have to think about the correlations anyway.

In the model with "AR", the autocorrelation coefficient indeed drifts towards 1. This can happen. Why this happens in this case is impossible to say for me without more information.

This aside, if 'random = list(~time|study, ~1|esid)', then there is only one '~ inner | outer' term, so the second "UN" in c("UN","UN") or c("AR","UN") is irrelevant. This doesn't affect anything but might suggest a misunderstanding of what the model is actually doing.

Best,
Wolfgang