Skip to content
Prev 26458 / 398502 Next

acceptable p-level for scientific studies

Kyriakos Kachrimanis (et al.) wrote:

            
In general, setting arbitrary criteria for statistical significance seems 
to be based upon a compromise between apparent progress (maximal 
discovery) and theoretical durability (minimal disconfirmation). If we are 
to build knowledge from ignorance or misapprehension, it is best to choose 
methods and criteria that lead to an optimal compromise. Statistical 
evaluation of data has done a much better job than rhetorical contention 
as a method. 
Criteria range from the apparently slack alpha=0.1 in fields where is it 
difficult to discover any regularity to approximately 0.000000001 for 
establishing an effect at "six sigma" where variables are apparently well 
described and measurement is correspondingly precise.
In fact, what seems to happen is that researchers and reviewers find 
criteria that allow them to advance, at least apparently, at a certain 
rate. Thus my opinion is that a certain level of apparent progress is 
psychologically necessary in research, and those in the messier areas are 
willing to look a bit more foolish.

Jim