P value value for a large number of degree of freedom in lmer
On 24/11/2010, at 1:09 PM, Jonathan Baron wrote:
For the record, I have to register my disagreement. In the experimental sciences, the name of the game is to design a well-controlled experiment, which means that the null hypothesis will be true if the alternative hypothesis is false. People who say what is below, which includes almost everyone who responded to this post, have something else in mind. What they say is true in most disciplines. But when I hear this sort of thing, it is like someone is telling me that my research career as an EXPERIMENTAL psychologist has been some sort of delusion. If you have a very large sample and you are doing a correlational study, yes, everything will be significant. But if you do the kind of experiment we struggle to design, with perfect control conditions, you won't get significant results (except by chance) if your hypothesis is wrong.
I'll bet you don't work with samples of size 200,000. :-) Also I'll bet that you don't ***really*** care if the difference between mu_T and mu_C is bigger than 0.000001 mm, say, whereas you might care if the difference were bigger than 10 mm. Also there's no such thing as ``perfect'' anything, let alone control conditions. cheers, Rolf Turner
Jon On 11/24/10 07:59, Rolf Turner wrote:
It is well known amongst statisticians that having a large enough data set will result in the rejection of *any* null hypothesis, i.e. will result in a small p-value. There is no ``bias'' involved.