Validation of R

From: Paul, David A [mailto:paulda at BATTELLE.ORG]

For software, our internal standards basically say that

(1) COTS (Com'l Off The Shelf software) developed by a
company having both a long history of selling high-quality
products and good QA doesn't need extensive from-scratch
validation, only validation of simpler routines like the
computation of means, variances, linear regression models,
&etc.  (After all, how would anyone really validate what,
say, PROC NLMIXED yields in a complex growth-curve application?)

Too bad that can't be edited just a bit:

(1) OTS (Off the Shelf software) developed by a group having both a
long history of creating high-quality products and good QA which
doesn't need extensive from-scratch validation, only validation of
simpler routines like the computation of means, variances, linear
regression models, etc.  (After all, how would anyone really validate
what, say, nlme(), yields in a complex growth-curve application, other
than one of the originators of one of the families of NLME
algorithms?)

I think this active and fun thread just demonstrates the futility and 
essential speciousness of the QA and other 'audit culture' processes when 
it comes to science.

There are multiple sources of 'error' in analysis, and choosing an 
inappropriate method/model is by far the most likely, most misleading, and 
most pernicious ... and one entirely outside QA protocols.

Wrt COTS - MS is a large firm with a long history, and a substantial record 
for producing large (detractors would say 'bloated') and complex code that 
does all sorts of complex manipulations of binary strings.  I am sure the 
MS organisation will have comprehensive internal QA processes in 
place.  ... but would many of us use MS Excel(t)  to conduct even 'simple' 
statistical analyses?  And if not, why not?

Validation should require transparency of algorithm choice and 
implementation so all steps and assumptions of the mechanical process can 
be evaluated independently ...

Otherwise it is just a piece of paper,


Richard Rowe
Senior Lecturer
Department of Zoology and Tropical Ecology, James Cook University
Townsville, Queensland 4811, Australia
fax (61)7 47 25 1570
phone (61)7 47 81 4851
e-mail: Richard.Rowe at jcu.edu.au
http://www.jcu.edu.au/school/tbiol/zoology/homepage.html