-----Original Message-----
From: Michael Dewey [mailto:lists at dewey.myzen.co.uk]
Sent: Wednesday, 09 December, 2020 17:47
To: Valeria Ivaniushina; Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: [R-meta] rma, sandwich correction and very small data sets
Dear Valeria
I think as a general principle you are entitled to do your analysis even
on a small data-set as long as you accept that your results may not be
very precise. There seems to be a general feeling among analysts in the
area in which I work (health) that looking for small study effects is
not worth trying with fewer than ten studies and even with more may well
be uninformative. I am personally rather sceptical about identifying
observations as outliers in the absence of a scientific reason for doing so.
Michael
On 09/12/2020 15:21, Valeria Ivaniushina wrote:
Dear Wolfgang,
Thank you VERY much!
Thank you for correcting my code -- indeed, random effect on the 1st level
is totally needed!
A couple more questions, if I may
1. There are too little cases for such a complex data structure, and it's
serious limitation.
But I hope that even if the results may be considered only as descriptive,
they still point out in the correct direction?
Especially taking into account that all three subsamples show quite
results.
Is it a valid interpretation?
2. Considering that the sample is small (and 3-level!), I guess that
analysis of outliers would be excessive. Is it right?
3. The same goes for publication bias analysis? (as James points out,
tests do not have strong power:
www.jepusto.com/publication/selective-reporting-with-dependent-effects/ )
4. and there is no power for mediation analysis, so I don't have to even
attempt to do it?
5. Estimators question:
"robust" function in rma is using sandwich-type estimator, and with
= TRUE it does a small-sample adjustment
In the clubSandwich library there are a bunch of estimators with different
small sample corrections. They give somewhat different results, some are
very close to "robust" output
Is clubSandwich CR2 (for example) better than robust.rma?
Or, if CR estimators from clubSandwich are not definitely preferable, can
just use robust.rma?
Best,
Valeria