Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
Power analysis for MANOVA?
9 messages · Stephan Kolassa, Mitchell Maltenfort, Adam D. I. Kramer +1 more
Hi Adam, My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy. HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
http://www.amazon.com/Statistical-Power-Analysis-Behavioral-Sciences/dp/0805802835 Cohen's book was in fact the basis for the "pwr" package at CRAN. And it does have a MANOVA power analysis, which was left out of the "pwr" package.
On Mon, Jan 26, 2009 at 4:12 PM, Adam D. I. Kramer <adik at ilovebacon.org> wrote:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Due to the recession, requests for instant gratification will be deferred until arrears in scheduled gratification have been satisfied.
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
If you know what a 'general linear hypothesis test' is see http://cran.r-project.org/src/contrib/Archive/hpower/hpower_0.1-0.tar.gz HTH, Chuck
On Mon, 26 Jan 2009, Adam D. I. Kramer wrote:
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple
case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Charles C. Berry (858) 534-2098
Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901
On Mon, 26 Jan 2009, Charles C. Berry wrote:
If you know what a 'general linear hypothesis test' is see http://cran.r-project.org/src/contrib/Archive/hpower/hpower_0.1-0.tar.gz
I do, and am quite interested, however this package will not install on R
2.8.1: First, it said that there was no "maintainer" in the description, so
I added one (figuring that the 1991 date of the package was to blame),
however it still will not compile:
parmesan:tmp$ sudo R CMD INSTALL hpower/
* Installing to library '/usr/local/lib/R/library'
* Installing *source* package 'hpower' ...
** R
** preparing package for lazy loading
Error in parse(n = -1, file = file) : unexpected '{' at
5: ##
6: pfnc_function(q,df1,df2,lm,iprec=c(6)) {
Calls: <Anonymous> -> code2LazyLoadDB -> sys.source -> parse
Execution halted
ERROR: lazy loading failed for package 'hpower'
** Removing '/usr/local/lib/R/library/hpower'
parmesan:tmp$
...any tips?
--Adam
HTH, Chuck On Mon, 26 Jan 2009, Adam D. I. Kramer wrote:
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple >
case
of 5 dependent variables and a single two-level categorical predictor (though the categories aren't balanced).
If anybody happens to know of a script that will do this in R,
I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though
of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide >
http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Charles C. Berry (858) 534-2098
Dept of Family/Preventive
Medicine
E mailto:cberry at tajo.ucsd.edu UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901
On Mon, 26 Jan 2009, Adam D. I. Kramer wrote:
On Mon, 26 Jan 2009, Charles C. Berry wrote:
If you know what a 'general linear hypothesis test' is see http://cran.r-project.org/src/contrib/Archive/hpower/hpower_0.1-0.tar.gz
I do, and am quite interested, however this package will not install on R
2.8.1: First, it said that there was no "maintainer" in the description, so
I added one (figuring that the 1991 date of the package was to blame),
however it still will not compile:
parmesan:tmp$ sudo R CMD INSTALL hpower/
* Installing to library '/usr/local/lib/R/library'
* Installing *source* package 'hpower' ...
** R
** preparing package for lazy loading
Error in parse(n = -1, file = file) : unexpected '{' at
5: ##
6: pfnc_function(q,df1,df2,lm,iprec=c(6)) {
_________^_________ AHA! That underscore is the old 'assignment' operator - now no longer allowed. Do a global replace of '_' with ' <- ' in the R/*.R files and it should install. HTH, Chuck
Calls: <Anonymous> -> code2LazyLoadDB -> sys.source -> parse Execution halted ERROR: lazy loading failed for package 'hpower' ** Removing '/usr/local/lib/R/library/hpower' parmesan:tmp$ ...any tips? --Adam
HTH, Chuck On Mon, 26 Jan 2009, Adam D. I. Kramer wrote:
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method
to
conduct a power analysis for a MANOVA. My interest is a fairly simple >
case
of 5 dependent variables and a single two-level categorical predictor (though the categories aren't balanced).
If anybody happens to know of a script that will do this in
R,
I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful,
though
of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide >
http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Charles C. Berry (858) 534-2098
Dept of Family/Preventive
Medicine
E mailto:cberry at tajo.ucsd.edu UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901
Charles C. Berry (858) 534-2098
Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901
1 day later
Hi Adam, first: I really don't know much about MANOVA, so I sadly can't help you without learning about it an Pillai's V... which I would be glad to do, but I really don't have the time right now. Sorry! Second: you seem to be doing a kind of "post-hoc power analysis", "my result isn't significant, perhaps that's due to low power? Let's look at the power of my experiment!" My impression is that "post-hoc power analysis" and its interpretation is, shall we say, not entirely accepted within the statistical community, see: Hoenig, J. M., & Heisey, D. M. (2001, February). The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55 (1), 1-6 And this: http://staff.pubhealth.ku.dk/~bxc/SDC-courses/power.pdf However, I am sure that lots of people can discuss this more competently than me... Best wishes Stephan Adam D. I. Kramer schrieb:
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple
case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
1 day later
Thanks for the response, Stephan. Really, I am trying to say, "My result is insignificant, my effect sizes are tiny, you may want to consider the possibility that there really are no meaningful differences." Computing post-hoc power makes a bit stronger of a claim in this setting. My real goal in this case was to put a single line on a poster that says, "Significance using our estimates would require N observations, which is larger than the population." I am trying to solve for N. N in this case is a sort of effect size. In this case, it is indeed a simple transformation of Pillai's V and the p-value for the study, and I do not intend to suggest that it is anything more than that. However, I believe that the latter effect size makes a much more compelling case, given that a lot of people (such as yourself) don't have much experience with Pillai's V. --Adam
On Wed, 28 Jan 2009, Stephan Kolassa wrote:
Hi Adam, first: I really don't know much about MANOVA, so I sadly can't help you without learning about it an Pillai's V... which I would be glad to do, but I really don't have the time right now. Sorry! Second: you seem to be doing a kind of "post-hoc power analysis", "my result isn't significant, perhaps that's due to low power? Let's look at the power of my experiment!" My impression is that "post-hoc power analysis" and its interpretation is, shall we say, not entirely accepted within the statistical community, see: Hoenig, J. M., & Heisey, D. M. (2001, February). The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55 (1), 1-6 And this: http://staff.pubhealth.ku.dk/~bxc/SDC-courses/power.pdf However, I am sure that lots of people can discuss this more competently than me... Best wishes Stephan Adam D. I. Kramer schrieb:
On Mon, 26 Jan 2009, Stephan Kolassa wrote:
My (and, judging from previous traffic on R-help about power analyses, also some other people's) preferred approach is to simply simulate an effect size you would like to detect a couple of thousand times, run your proposed analysis and look how often you get significance. In your simple case, this should be quite easy.
I actually don't have much experience running monte-carlo designs like this...so while I'd certainly prefer a bootstrapping method like this one, simulating the effect size given my constraints isn't something I've done before. The MANOVA procedure takes 5 dependent variables, and determines what combination of the variables best discriminates the two levels of my independent variable...then the discrimination rate is represented in the statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71). So coming up with a set of constraints that would produce V=.00019 given my data set doesn't quite sound trivial...so I'll go for the "par" library reference mentioned earlier before I try this. That said, if anyone can refer me to a tool that will help me out (or an instruction manual for RNG), I'd also be much obliged. Many thanks, Adam
HTH, Stephan Adam D. I. Kramer schrieb:
Hello,
I have searched and failed for a program or script or method to
conduct a power analysis for a MANOVA. My interest is a fairly simple
case
of 5 dependent variables and a single two-level categorical predictor
(though the categories aren't balanced).
If anybody happens to know of a script that will do this in R, I'd
love to know of it! Otherwise, I'll see about writing one myself.
What I currently see is this, from help.search("power"):
stats::power.anova.test
Power calculations for balanced one-way
analysis of variance tests
stats::power.prop.test
Power calculations two sample test for
proportions
stats::power.t.test Power calculations for one and two sample t
tests
Any references on power in MANOVA would also be helpful, though of
course I will do my own lit search for them myself.
Cordially,
Adam D. I. Kramer
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.