Validation of R

-----Original Message-----
From: Paul, David A [mailto:paulda at BATTELLE.ORG]
Sent: Thursday, April 17, 2003 4:18 PM
To: r-help at stat.math.ethz.ch
Subject: RE: [R] Validation of R


At Battelle, the QA/QC folks have the philosophy that the
FDA will likely hold us responsible for whatever internal
standards we set for ourselves, assuming that such standards
are "reasonable".

For software, our internal standards basically say that

(1) COTS (Com'l Off The Shelf software) developed by a
company having both a long history of selling high-quality
products and good QA doesn't need extensive from-scratch
validation, only validation of simpler routines like the
computation of means, variances, linear regression models,
&etc.  (After all, how would anyone really validate what, 
say, PROC NLMIXED yields in a complex growth-curve application?)

(2) Anything free needs to be extensively validated by 
comparing it with something that fits into (1)

This leaves R completely out of our GLP studies, and favors
SAS since Insightful hasn't been around as long as the SAS 
Institute.  Like it or not, the perception is that using SAS
won't get you into trouble with the FDA or other regulatory
agencies.


-David Paul



-----Original Message-----
From: partha_bagchi at hgsi.com [mailto:partha_bagchi at hgsi.com] 
Sent: Thursday, April 17, 2003 3:32 PM
To: Frank E Harrell Jr
Cc: k.benjamin at retroscreen.com; r-help at stat.math.ethz.ch;
a.mann at retroscreen.com; s.balasingam at retroscreen.com;
r.lambkin at retroscreen.com; v_bill_pikounis at merck.com;
s.bossuyt at retroscreen.com
Subject: Re: [R] Validation of R


I agree with your points and if you notice I share your philosophical 
view. I was commenting more on what you call "mind" share. It 
is still 
real. 

However, also a minor point  - there is mention in the regs 
regarding COTS 
software (which I believe stands for Commercial of the Shelf 
software) ..






Frank E Harrell Jr <fharrell at virginia.edu>
04/17/2003 12:10 PM

 
        To:     partha_bagchi at hgsi.com
        cc:     v_bill_pikounis at merck.com, 
k.benjamin at retroscreen.com, 
r-help at stat.math.ethz.ch, a.mann at retroscreen.com, 
s.balasingam at retroscreen.com, r.lambkin at retroscreen.com, 
s.bossuyt at retroscreen.com
        Subject:        Re: [R] Validation of R


On Thu, 17 Apr 2003 10:38:06 -0400
partha_bagchi at hgsi.com wrote:

However, the perception out there is the "SAS is the accepted 
software" especially for regulatory submission and 

especially in the 

US. Thus, I think validation usually means "Yeah, but did 

you use SAS 

to get the answer" , no matter how irrelevant the question 

is. For a 

non-statistician, or a person doing validation certain 

software do not 

need validation (Microsoft Word, SAS etc.) certain other , perhaps 
more

so

for open source, validation is essential.

SAS is NOT the accepted software for FDA, because FDA does 
not accept ANY 
brand of software.  This is really a "mind share" issue at pharma 
companies.  SAS is not validated in every sense; there is a 
huge list of 
current SAS bugs.

Validation is best done on a per-project basis as you can't 
anticipate all 
aspects of a particular dataset.  The validation can be done by 
independent calculations of pivotal findings.  For R there is an 
especially good opportunity because if you are using the base 
packages you 
can run essentially the same code in S-Plus to get an independent 
validation of the underlying calculations (but not of your S 
code).  The 
base code in R is independent of that in S-Plus (this is not 
true of most 
add-on packages by users).  There is no other "SAS" you can run.

---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences U.
Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



	[[alternate HTML version deleted]]