[R-meta] [External] Re: Do transformations of omnibus statistical tests show the correct direction of effect in the effect size values?
Dear James, Thanks so much for your response. I have done exactly that as I have been working. It is a relief to me to know that this is a safe way to proceed. Many thanks. Emma ----------------------------------------------------------------------------------------------- Emma Mills PhD Student Psychology Department, Lancaster University, Lancaster, LA1 4YF Email: e.mills at lancaster.ac.uk; Telephone: +44 794 353 5855 twitter: EmmaMil26429683 Email too brief? Here's why! http://emailcharter.org
From: James Pustejovsky <jepusto at gmail.com>
Sent: 12 January 2021 15:13
To: Mills, Emma (Student) <e.mills at lancaster.ac.uk>
Cc: r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>
Subject: [External] Re: [R-meta] Do transformations of omnibus statistical tests show the correct direction of effect in the effect size values?
Sent: 12 January 2021 15:13
To: Mills, Emma (Student) <e.mills at lancaster.ac.uk>
Cc: r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>
Subject: [External] Re: [R-meta] Do transformations of omnibus statistical tests show the correct direction of effect in the effect size values?
This email originated outside the University. Check before clicking links or attachments. Hi Emma, I would advise to carefully inspect your data and computed effect sizes to determine whether you need to make a further sign-transformation. For my own projects, I do this by creating tables (in Rmarkdown or by exporting to Excel) containing the raw summary statistics, direction of effect, and computed effect size, organized in a way to make it easy to check (e.g., by sorting the studies on direction of effect). Then I read through the table, looking for potential inconsistencies. Even when using very well-validated software, I think this is a good general habit to build. Kind Regards, James On Tue, Jan 12, 2021 at 8:31 AM Mills, Emma (Student) <e.mills at lancaster.ac.uk<mailto:e.mills at lancaster.ac.uk>> wrote: Dear Meta-Analysis Forum I should be very grateful if you could give me any guidance re the question below. I've looked back in the archives over the past year but do not see anything that looks like this question. I am a first time poster, so any other advice for making my posts more effective also greatly appreciated. The data I have extracted for my meta-analysis includes F-ratio values, p-values, t-test values - where the original papers have omitted means and / or sd values. Along with those values, I have collected the direction of the effect from the report narrative. I have used esc? / compute.es<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcompute.es%2F&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301083433%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vq5EVW5SZysXE4SbKlRnzROnx2ulptoQUc4sAk%2Fy9Y4%3D&reserved=0>? R packages to transform the collected data to either Hedges' g, log(odds) or fisher's z. What I am not confident of is that the transformation of omnibus statistic values includes the appropriate direction of the effect, in the way that deriving the effect sizes from means so clearly does. Should I be confident of the transformation? Or should I multiply the new value by -1 / 1 to ensure the direction of the effect is captured in the individual effect size and then continue with the meta-analysis? Many thanks in anticipation of your response. I hope you remain well during these strange times. Emma Mills ----------------------------------------------------------------------------------------------- Emma Mills PhD Student Psychology Department, Lancaster University, Lancaster, LA1 4YF Email: e.mills at lancaster.ac.uk<mailto:e.mills at lancaster.ac.uk>; Telephone: +44 794 353 5855 twitter: EmmaMil26429683 Email too brief? Here's why! http://emailcharter.org<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Femailcharter.org%2F&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301083433%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=y8u5AXnkXyTzcq1ywpoCcviY8b8Fgw8c5PJq3kuyMoE%3D&reserved=0> _______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org<mailto:R-sig-meta-analysis at r-project.org> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301093423%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FIJP05dbpbFd%2B8MfgqBoZhR5fQ3ve%2FgyREMN421eKco%3D&reserved=0>