[R-meta] metagen / low heterogeneity
On 11/01/2021 16:45, Sean wrote:
I apologize for the formatting. Here is the ouput and code again below. I think this should be more readable now that I've selected plain text. Michael, well that is good news. If I did have high heterogeneity and hadn't planned to use a moderator, does that just mean I should consider looking for one? Whereas in my case, I knew what I was interested in, so my heterogeneity does not need to be considered as a prerequisite?
The crucial thing is the scientific context. I do not work in the same area as you so my examples are from my field, not yours, but I hope are helpful. If the primary studies were all very similar then you would not expect heterogeneity and you might be prompted to look for explanations for even mild amounts. For instance if all the primary studies had studied the same dose of drug in people with very tightly defined illness in countries with very similar health care symptoms then any heterogeneity might lead you, post hoc, to find out why. If on the contrary the studies had examined a complex health care systems intervention in countries across the globe in patients who might vary considerably then you would be very surprised not to see heterogeneity. In that case you would be less inclined to look for explanations. If you had a theory that outcomes were related to some other variable then you might use that as a moderator irrespective of the amount of heterogeneity. For instance in a study of a skills-based therapy you might have a theory that outcomes are different now from what they used to be so you would find it worth while looking at that whatever. For instance is centres in each study have been doing a particular operation for different amounts of time do the ones who have been doing it for longest have have better or worse outcomes. Michael
Here is an example of my output:
Number of studies combined: k = 288
SMD 95%-CI t
p-value
Random effects model 0.3309 [ 0.2866; 0.3751] 14.72 < 0.0001
Prediction interval [-0.2216; 0.8834]
Quantifying heterogeneity:
tau^2 = 0.0783 [<0.0000; <0.0000]; tau = 0.2798 [<0.0000; <0.0000];
I^2 = 0.0% [0.0%; 0.0%]; H = 1.00 [1.00; 1.00]
Test of heterogeneity:
Q d.f. p-value
165.46 287 1.0000
Here is the code:
metamkt <- metagen(G,
seG,
data = mkt,
studlab = paste(Study),
comb.fixed = FALSE,
comb.random = TRUE,
method.tau = "SJ",
hakn = TRUE,
prediction = TRUE,
sm = "SMD")
Sean
On Mon, Jan 11, 2021 at 11:11 AM Michael Dewey <lists at dewey.myzen.co.uk> wrote:
Dear Sean Some comments in-line. It is difficult to read your output because you posted in HTML so I will leave that to people more familiar with the software. Next time it would help to set your mailer to use plain text so your message does not get mangled. On 11/01/2021 14:56, Sean wrote:
Hello Meta-analysis Community, I've been using the metagen function in the meta package for a meta-analysis on fungicide efficacy to control a foliar pathogen in cucumbers. I'm using pre-calculated Hedge's G as my effect size and it's standard error. I'm not really a statistician, so I've been using this resource to hold my hand through the process ( https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/random.html). I've run into a bit of a rut and I'm having a hard time getting help to interpret my results. I'm dealing with the issue of some of my dataset heterogeneity being nearly 0 (which could just be the case). *Here is an example of my output:* Number of studies combined: k = 288 SMD 95%-CI t p-value Random effects model 0.3309 [ 0.2866; 0.3751] 14.72 < 0.0001 Prediction interval [-0.2216; 0.8834]
The fact that your prediction interval is so much wider than the confidence interval does suggest there is heterogeneity here.
Quantifying heterogeneity:
tau^2 = 0.0783 [<0.0000; <0.0000]; tau = 0.2798 [<0.0000; <0.0000];
I^2 = 0.0% [0.0%; 0.0%]; H = 1.00 [1.00; 1.00]
Test of heterogeneity:
Q d.f. p-value
165.46 287 1.0000
*Here is the code:*
metamkt <- metagen(G,
seG,
data = mkt,
studlab = paste(Study),
comb.fixed = FALSE,
comb.random = TRUE,
method.tau = "SJ",
hakn = TRUE,
prediction = TRUE,
sm = "SMD")
My first red flag is of course "I^2 = 0.0%", then that my Q p-value is 1.
The interpretation being that the observed heterogeneity is completely
random. I have a couple datasets, with the highest I^2 = 17.4%. The reason
I find it odd, is that when I do subgroup analysis (even though I'm not
supposed to with such low / non-existat heterogeneity), the results make
biological sense.
No, no, a thousand times no. You use a moderator if there is a scientific hypothesis which justifies it not because of observed heterogeneity. In this case if there is a biological theory behind a moderator then use it. Michael My data spans the last decade and the results are also
similar with a meta-analysis done in the previous decade on the same topic.
This makes me feel like I've made some sort of error at some point in my
workflow and I was wondering if you have any diagnostic recommendations for
me? One thing that worries me is that my standard errors for my Hedge's G
values are so similar since all treatments in each study have 4
replications, but maybe it shouldn't.
Best,
Sean
[[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
-- Michael http://www.dewey.myzen.co.uk/home.html