[R-meta] Subgroup analysis output using metafor - interpretation

Dear Joao

I hink we may need some clarification before we can answer this. 
Comments in-line below

Dear all,

I am running a meta-analysis on the prevalence of lameness (binary) in
British dairy cattle and have used the *metaprop* from the *metafor* package.

I think metaprop comes from meta, not metafor?

I have set the model to run with random effects, using the DL method, and
have taken the following approach:

    1. log-transform the data as it is not normally distributed

If it is binary data I would not have expected that anyway so what 
exactly did you transform?

    2. identify outliers using influential analysis (only ran this once)
    3. remove outliers and rerun the model

In general that seems a bad idea as it removes the most interesting 
observations but you may have reasons to doubt the observations.

    4. deal with remaining heterogeneity with subgroup analysis and
    meta-regression

I have ran the model and am getting what I believe conflicting evidences on
different output indicators. As an example, after running subgroup analysis
with one moderator, the output tells me that the moderator explains around
50% of the heterogeneity (R^2), and yet the p-value for the test of
moderators is substantially higher than 0.05 telling me that the pooled
estimates of the subgroups aren't actually different.

Can you share the output from the analysis to give us a clue?

I was hoping you could shed a light as to what could justify this happening
(if it makes sense), and possibly provide some guidance as to what I could
do to improve the statistical evidences of my study.


Many thanks and happy 2020 to everyone

Michael
http://www.dewey.myzen.co.uk/home.html

Dear Michael,

Thank you for your prompt reply. Sorry for my eventual mistake with
the *metaprop
*command. It is likely that it comes from the *meta* package. I had to
install it as well as the *metafor* package, to run model.

I have log-transformed the number of *events *and *n *using logit
transformed proportion:

*ies.logit=escalc(xi=nlameanimal, ni=ssizeanimal,
data=prevalence_2020_nomv, measure="PLO")*
*pes.logit=rma(yi, vi, data=ies.logit, method = "DL")*
*pes=predict(pes.logit, transf=transf.ilogit)*
*print(pes.logit, digits=4)*

As for the outliers I could take a step back and instead of removing them
leave them in the data-set and see what happens when conducting the
sub-group analysis. Is this best practice when conducing a meta-analysis?

Outputs below


   - Output from meta-analysis after removing outliers


* Random-Effects Model (k = 42; tau^2 estimator: DL)*



* tau^2 (estimated amount of total heterogeneity): 0.2307 (SE = 0.1768)*

* tau (square root of estimated tau^2 value): 0.4803*

* I^2 (total heterogeneity / total variability): 99.73%*

* H^2 (total variability / sampling variability): 372.29*



* Test for Heterogeneity:*

* Q(df = 41) = 15263.8748, p-val < .0001*



* Model Results:*



* estimate se zval pval ci.lb <http://ci.lb> ci.ub*

* -0.8445 0.0802 -10.5351 <.0001 -1.0016 -0.6874 ****



* ---*

* Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1*

*confint(pes.logit, digits=2)*



* estimate ci.lb <http://ci.lb> ci.ub*

* tau^2 0.23 0.60 1.68*

* tau 0.48 0.77 1.30*

* I^2(%) 99.73 99.90 99.96*

* H^2 372.29 963.40 2707.90*

*print(pes, digits=4)*



* pred ci.lb <http://ci.lb> ci.ub cr.lb <http://cr.lb> cr.ub*

* 0.3006 0.2686 0.3346 0.1420 0.5274*




   - Output from subgroup analysis with lcmbi as moderator:


 *Mixed-Effects Model (k = 42; tau^2 estimator: DL)*



* tau^2 (estimated amount of residual heterogeneity): 0.0977 (SE = 0.0677)*

* tau (square root of estimated tau^2 value): 0.3125*

* I^2 (residual heterogeneity / unaccounted variability): 99.17%*

* H^2 (unaccounted variability / sampling variability): 120.25*

* R^2 (amount of heterogeneity accounted for): 57.67%*



* Test for Residual Heterogeneity:*

* QE(df = 40) = 4809.8552, p-val < .0001*



* Test of Moderators (coefficient 2):*

* QM(df = 1) = 1.5762, p-val = 0.2093*



* Model Results:*



* estimate se zval pval ci.lb <http://ci.lb> ci.ub*

* intrcpt -0.8191 0.0625 -13.1124 <.0001 -0.9416 -0.6967 ****

* lcmbiRecords -0.1653 0.1317 -1.2555 0.2093 -0.4234 0.0928*



* ---*

* Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1*

*print(pes.subg.lcmbi[1], digits=6)*



* pred ci.lb <http://ci.lb> ci.ub cr.lb <http://cr.lb> cr.ub*

* 1 0.305946 0.280581 0.332544 0.190968 0.451515*

*print(pes.subg.lcmbi[17], digits=6)*



* pred ci.lb <http://ci.lb> ci.ub cr.lb <http://cr.lb> cr.ub*

* 17 0.305946 0.280581 0.332544 0.190968 0.451515*

*print(pes.lcmbi, digits=6)*



* pred ci.lb <http://ci.lb> ci.ub cr.lb <http://cr.lb> cr.ub*

* 0.294637 0.264094 0.327143 0.253791 0.339071*





Many thanks for all the help Michael



On Thu, Jan 9, 2020 at 1:57 PM Michael Dewey <lists at dewey.myzen.co.uk>
wrote:

Dear Joao

I hink we may need some clarification before we can answer this.
Comments in-line below

On 09/01/2020 13:36, Joao Afonso wrote:

Dear all,

I am running a meta-analysis on the prevalence of lameness (binary) in
British dairy cattle and have used the *metaprop* from the *metafor*

package.

I think metaprop comes from meta, not metafor?

I have set the model to run with random effects, using the DL method, and
have taken the following approach:

    1. log-transform the data as it is not normally distributed

If it is binary data I would not have expected that anyway so what
exactly did you transform?

    2. identify outliers using influential analysis (only ran this once)
    3. remove outliers and rerun the model

In general that seems a bad idea as it removes the most interesting
observations but you may have reasons to doubt the observations.

    4. deal with remaining heterogeneity with subgroup analysis and
    meta-regression

I have ran the model and am getting what I believe conflicting evidences

on

different output indicators. As an example, after running subgroup

analysis

with one moderator, the output tells me that the moderator explains

around

50% of the heterogeneity (R^2), and yet the p-value for the test of
moderators is substantially higher than 0.05 telling me that the pooled
estimates of the subgroups aren't actually different.

Can you share the output from the analysis to give us a clue?

I was hoping you could shed a light as to what could justify this

happening

(if it makes sense), and possibly provide some guidance as to what I

could

do to improve the statistical evidences of my study.


Many thanks and happy 2020 to everyone

--
Michael
http://www.dewey.myzen.co.uk/home.html

Jo?o Afonso
*DVM, MSc Veterinary Epidemiology*
*PhD Student *
*Department of Infection and Global Health*
*University of Liverpool*
*+351914812305*

	[[alternative HTML version deleted]]

Am 09.01.20 um 15:26 schrieb Joao Afonso:

[...]
As for the outliers I could take a step back and instead of removing them
leave them in the data-set and see what happens when conducting the
sub-group analysis. Is this best practice when conducing a meta-analysis?

I am no expert on outlier detection (in meta-analysis), however, I would 
say that it is in general a good idea to try to explain between-study 
heterogeneity using subgroup analysis or meta-regression first.

An intrinsic feature / problem of the meta-analysis of prevalences is 
typically the substantial between-study heterogeneity (which is clearly 
visible in your meta-analysis with an I2 of 99.7%). In my view, the main 
aim of a prevalence meta-analysis is actually to describe this 
heterogeneity. Removing outliers would reduce this heterogeneity and 
paint a different picture. I assume that there will be still substantial 
heterogeneity after conducting subgroup analyses. Prediction intervals 
can help to properly describe the between-study heterogeneity.

Best wishes, Guido