Skip to content
Prev 200551 / 398503 Next

Normal distribution test

On 11/17/2009 10:07 AM, rkevinburton at charter.net wrote:
I think you have an extra "not" in your description of the null.  But 
more importantly, a small p-value just means that you had a value of the 
test statistic that would be unusually extreme if the null was true.

Whether that means there is a small probability of the null being true 
depends on information outside the test.  It's essentially a Bayesian 
question, and Bayesians wouldn't base their decision on just the p-value.

For example, if I flip a fair coin 100 times and see 30 heads, 
binom.test() tells me that the p value would be 7.85e-05.  That doesn't 
mean the probability that the coin is fair is low:  it just means I saw 
an unusually small number of heads.

Now if I wasn't sure whether the coin was fair or not, I'd take the low 
p-value as evidence that it was not.  But the p-value alone isn't 
sufficient to let me calculate a probability of the null being true.

Duncan Murdoch