Skip to content

F test vs. mcmcpvalue

2 messages · Ben Bolker, Ken Beath

#
Hank Stevens wrote:
d> HI Ben and Spencer,
| Thank you very much for your help.
|
| 1. The QQ plots look normal, but highlight the lack of balance (from one
| to dozens of reps per treatment combo).
| 2. The MCMC sample traces look (in my limited experience) without
| peculiarities, and the densityplots are all quite symmetrical and
| normal-ish.
| 3. Simulations (lmer::simulate) of the null hypothesis indicate that
| F-stats as large (or larger) than my observed F-stats are VERY unlikely,
| under the null hypothesis.
|
| As I learn anything else useful, I will be happy to share.
| Cheers,
| Hank
|

~  #3 pretty much seals it for me -- since that is really what
the F test is trying to test.

~  It's a little hard to reconcile #2 and #3, though ... I would
think you could move on at this point, but just for laughs --
are you using mcmcpvalue on a single contrast, or multiple
parameters?  If the former, does it seem to agree with the results of
HPDinterval() or quantile()?  If the latter, is there something
about the _combinations_ of parameters that is wonky?

~  Don't forget my earlier comment about whether what you are testing
(p-values of main effects in the presence of interactions) actually
makes sense ...

~  cheers
~   Ben
1 day later
#
On 08/07/2008, at 3:16 AM, Ben Bolker wrote:

            
If an MCMC isn't traversing the parameter space properly the traces  
will probably still look OK until it shifts into a new region which  
may take a while.

Also it was mentioned that there were 500 observations. For clustered  
data it is the number of clusters that is more important.

Ken