Correlating multiple effect sizes within a study to study-level predictors: metafor package

Hi everyone,

Since metafor doesn't have its own list, I hope this is the correct place
for this posting- my apologies if there is a more appropriate list.

I'm conducting a meta-analysis where I would like to determine the
correlation between plasticity in leaf traits and climate. I'm calculating
effect sizes as Hedge's d. My data is structured so that each study
collected data from one forest site, so there is one set of climate
variable values for that study, and there are one or more species in each
study, so all the species in a study have the same values for the climate
variables. I'm not sure how to account for this structure in modeling the
relationship between plasticity and climate.

My first thought was to calculate mean effect size and variance across
species for every study with multiple species and correlate that    with
the climate variable values for those study with the rma() function, but
trying to do that returns an error message:

rma(yi = EffectSize, vi = Var, data = sitestable, mod = Precip)
returns: Error in wi * (yi - X %*% b)^2 : non-conformable arrays

This leaves me with two questions: 1) Am I even accounting for the data
structure correctly with this approach, and 2) am I fundamentally
misunderstanding how to use metafor to do so?

Thanks very much for your help!

Best,

Megan

        [[alternative HTML version deleted]]

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Michael Dewey
Sent: Monday, July 14, 2014 14:42
To: Megan Bartlett; r-help at r-project.org
Subject: Re: [R] Correlating multiple effect sizes within a study to
study-level predictors: metafor package

At 23:18 11/07/2014, Megan Bartlett wrote:

Hi everyone,

Since metafor doesn't have its own list, I hope this is the correct

place

for this posting- my apologies if there is a more appropriate list.

metafor questions welcome here, Megan

Wolfgang seems to be off-list so while we wait for the definitive
answer here are some hints.

I'm conducting a meta-analysis where I would like to determine the
correlation between plasticity in leaf traits and climate. I'm

calculating

effect sizes as Hedge's d. My data is structured so that each study
collected data from one forest site, so there is one set of climate
variable values for that study, and there are one or more species in

each

study, so all the species in a study have the same values for the

climate

variables. I'm not sure how to account for this structure in modeling

the

relationship between plasticity and climate.

I think you need rma.mv for your situation and you need to specify a
random effect for site.

Try going
?rma.mv
and looking for the section entitled Specifying random effects
  You will need to set up your dataframe with one row per species and
an indicator variable for site and then use
random = ~ 1 | site

Not tested obviously and Wolfgang may have other suggestions

My first thought was to calculate mean effect size and variance across
species for every study with multiple species and correlate that    with
the climate variable values for those study with the rma() function, but
trying to do that returns an error message:

rma(yi = EffectSize, vi = Var, data = sitestable, mod = Precip)
returns: Error in wi * (yi - X %*% b)^2 : non-conformable arrays

This leaves me with two questions: 1) Am I even accounting for the data
structure correctly with this approach, and 2) am I fundamentally
misunderstanding how to use metafor to do so?

Thanks very much for your help!

Best,

Megan

Thanks very much, Wolfgang and Michael! I feel like I understand rma much
more clearly.

But just to make sure, is there any way to do this kind of analysis for a
continuous predictor variable?

For each site level, I have a value for a
climate variable, and it would be great to see whether the average effect
size for each site is correlated with that climate variable. But I'm not
sure what variance would produce the appropriate weighting for each
site-level average- would it be the variance in effect sizes across species
within each site? Or does this analysis not really make any sense for
effect sizes?

Thanks again!

Best,

Megan


On Mon, Jul 14, 2014 at 6:06 AM, Viechtbauer Wolfgang (STAT) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

Somehow that initial post slipped under the radar for me ...

Yes, I would give the same suggestion as Michael. Besides random effects
for 'site', I would also suggest to add random effects for each estimates
(as in a regular random-effects model). So, if you have an 'id' variable
that is unique to each observed d-value, you would use:

random = list(~ 1 | site, ~ 1 | id)

with the rma.mv() function. This is in essence the model given by
equation (6) in:

Nakagawa, S., & Santos, E. S. A. (2012). Methodological issues and
advances in biological meta-analysis. Evolutionary Ecology, 26(5),
1253-1274.

(at the time of publication, this model could not be fitted with metafor,
but it can now). Same model is described with a bit more detail in:

Konstantopoulos, S. (2011). Fixed effects and variance components
estimation in three-level meta-analysis. Research Synthesis Methods, 2(1),
61-76.

Best,
Wolfgang

--
Wolfgang Viechtbauer, Ph.D., Statistician
Department of Psychiatry and Psychology
School for Mental Health and Neuroscience
Faculty of Health, Medicine, and Life Sciences
Maastricht University, P.O. Box 616 (VIJV1)
6200 MD Maastricht, The Netherlands
+31 (43) 388-4170 | http://www.wvbauer.com

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Michael Dewey
Sent: Monday, July 14, 2014 14:42
To: Megan Bartlett; r-help at r-project.org
Subject: Re: [R] Correlating multiple effect sizes within a study to
study-level predictors: metafor package

At 23:18 11/07/2014, Megan Bartlett wrote:

Hi everyone,

Since metafor doesn't have its own list, I hope this is the correct

place

for this posting- my apologies if there is a more appropriate list.

metafor questions welcome here, Megan

Wolfgang seems to be off-list so while we wait for the definitive
answer here are some hints.

I'm conducting a meta-analysis where I would like to determine the
correlation between plasticity in leaf traits and climate. I'm

calculating

effect sizes as Hedge's d. My data is structured so that each study
collected data from one forest site, so there is one set of climate
variable values for that study, and there are one or more species in

each

study, so all the species in a study have the same values for the

climate

variables. I'm not sure how to account for this structure in modeling

the

relationship between plasticity and climate.

I think you need rma.mv for your situation and you need to specify a
random effect for site.

Try going
?rma.mv
and looking for the section entitled Specifying random effects
  You will need to set up your dataframe with one row per species and
an indicator variable for site and then use
random = ~ 1 | site

Not tested obviously and Wolfgang may have other suggestions

My first thought was to calculate mean effect size and variance across
species for every study with multiple species and correlate that    with
the climate variable values for those study with the rma() function, but
trying to do that returns an error message:

rma(yi = EffectSize, vi = Var, data = sitestable, mod = Precip)
returns: Error in wi * (yi - X %*% b)^2 : non-conformable arrays

This leaves me with two questions: 1) Am I even accounting for the data
structure correctly with this approach, and 2) am I fundamentally
misunderstanding how to use metafor to do so?

Thanks very much for your help!

Best,

Megan

        [[alternative HTML version deleted]]

Hi Michael,

Thank you! Just to clarify, in my question, I was thinking that in this
regression each study should be treated as one point, instead of each
species, so that each effect size x value has a unique climate y value. Is
that what the random= list(~1|Species, ~1|Site) argument is doing?

Thanks,

Megan


On Wed, Jul 16, 2014 at 1:53 AM, Michael Dewey <info at aghmed.fsnet.co.uk>
wrote:

At 23:19 14/07/2014, Megan Bartlett wrote:

Thanks very much, Wolfgang and Michael! I feel like I understand rma much
more clearly.

But just to make sure, is there any way to do this kind of analysis for a
continuous predictor variable?

Yes, just put it in as a moderator.

I am not sure I fully understand the rest of your question but the answer
may be that the weights are a property of the individual effect sizes

 For each site level, I have a value for a

climate variable, and it would be great to see whether the average effect
size for each site is correlated with that climate variable. But I'm not
sure what variance would produce the appropriate weighting for each
site-level average- would it be the variance in effect sizes across
species
within each site? Or does this analysis not really make any sense for
effect sizes?

Thanks again!

Best,

Megan


On Mon, Jul 14, 2014 at 6:06 AM, Viechtbauer Wolfgang (STAT) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

Somehow that initial post slipped under the radar for me ...

Yes, I would give the same suggestion as Michael. Besides random effects
for 'site', I would also suggest to add random effects for each

estimates

(as in a regular random-effects model). So, if you have an 'id' variable
that is unique to each observed d-value, you would use:

random = list(~ 1 | site, ~ 1 | id)

with the rma.mv() function. This is in essence the model given by
equation (6) in:

Nakagawa, S., & Santos, E. S. A. (2012). Methodological issues and
advances in biological meta-analysis. Evolutionary Ecology, 26(5),
1253-1274.

(at the time of publication, this model could not be fitted with

metafor,

but it can now). Same model is described with a bit more detail in:

Konstantopoulos, S. (2011). Fixed effects and variance components
estimation in three-level meta-analysis. Research Synthesis Methods,

2(1),

61-76.

Best,
Wolfgang

--
Wolfgang Viechtbauer, Ph.D., Statistician
Department of Psychiatry and Psychology
School for Mental Health and Neuroscience
Faculty of Health, Medicine, and Life Sciences
Maastricht University, P.O. Box 616 (VIJV1)
6200 MD Maastricht, The Netherlands
+31 (43) 388-4170 | http://www.wvbauer.com

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-

project.org]

On Behalf Of Michael Dewey
Sent: Monday, July 14, 2014 14:42
To: Megan Bartlett; r-help at r-project.org
Subject: Re: [R] Correlating multiple effect sizes within a study to
study-level predictors: metafor package

At 23:18 11/07/2014, Megan Bartlett wrote:

Hi everyone,

Since metafor doesn't have its own list, I hope this is the correct

place

for this posting- my apologies if there is a more appropriate list.

metafor questions welcome here, Megan

Wolfgang seems to be off-list so while we wait for the definitive
answer here are some hints.

I'm conducting a meta-analysis where I would like to determine the
correlation between plasticity in leaf traits and climate. I'm

calculating

effect sizes as Hedge's d. My data is structured so that each study
collected data from one forest site, so there is one set of climate
variable values for that study, and there are one or more species in

each

study, so all the species in a study have the same values for the

climate

variables. I'm not sure how to account for this structure in modeling

the

relationship between plasticity and climate.

I think you need rma.mv for your situation and you need to specify a
random effect for site.

Try going
?rma.mv
and looking for the section entitled Specifying random effects
  You will need to set up your dataframe with one row per species and
an indicator variable for site and then use
random = ~ 1 | site

Not tested obviously and Wolfgang may have other suggestions

My first thought was to calculate mean effect size and variance

across

species for every study with multiple species and correlate that

 with

the climate variable values for those study with the rma() function,

but

trying to do that returns an error message:

rma(yi = EffectSize, vi = Var, data = sitestable, mod = Precip)
returns: Error in wi * (yi - X %*% b)^2 : non-conformable arrays

This leaves me with two questions: 1) Am I even accounting for the

data

structure correctly with this approach, and 2) am I fundamentally
misunderstanding how to use metafor to do so?

Thanks very much for your help!

Best,

Megan

        [[alternative HTML version deleted]]

Michael Dewey
info at aghmed.fsnet.co.uk
http://www.aghmed.fsnet.co.uk/home.html

        [[alternative HTML version deleted]]