The national institute of standards and technology offers reference data sets and expected results for various statistical procedures using these data sets. From the web site:
"The purpose of this project is to improve the accuracy of statistical software by providing reference datasets with certified computational results that enable the objective evaluation of statistical software."
http://www.itl.nist.gov/div898/strd/
-------Original Message-------
From: "Pikounis, Bill" <v_bill_pikounis at merck.com>
Sent: 04/17/03 07:53 AM
To: 'Rob Lambkin' <r.lambkin at retroscreen.com>, r-help at stat.math.ethz.ch
Subject: RE: [R] Validation of R
Hi Rob,
We conduct contract virology research for large pharma companies. My
question is how do we validate this software? I wonder if anyone else
has had the problem and might be able to comment.
Notwithstanding the disclaimer automatically appended below by my "Big
Pharma" member IT dept on all my email sends, I have not had this problem,
or perhaps more accurately, not allowed it to be a problem when work I
have
done in R has made it into drug application filings and responses to FDA
and
European regulatory agencies.
"Validation" of software is an ill-defined concept, so I am afraid I
cannot
offer anything like a concrete "how-to", not would I be surprised if
anyone
else can. What I would like to suggest is to (1) ask your vendor
companies
what specifically they are concerned about, (2) benchmark some guidelines
on
how you all or others have "validated" other software.
If you are looking for extensive documentation on whats/hows/whys of R, it
already has it. If you are looking for it to compute the same values as
"validated" software within realistic numeric accuracy for your
procedures,
that is straightforward to do. And the ultimate key is that anyone can
look
at the source code and have a high probability to get it to run on any
reasonably current system, and even many systems not so current.
On a visible, continuous (daily), *OPEN* basis, there is ongoing review
and
input from the R user community, as well as all the highest standards of
software engineering that are met by the R core team and other developers.
R
clearly stands up to rigorous, scholastic scrutiny. In my very grateful
view, this makes R at least as reliable as commercial vendor software that
claims "validation" or "compliance", etc., ...and probably, more reliable.
Hope that helps.
Bill
----------------------------------------
Bill Pikounis, Ph.D.
Biometrics Research Department
Merck Research Laboratories
PO Box 2000, MailDrop RY84-16
126 E. Lincoln Avenue
Rahway, New Jersey 07065-0900
USA
v_bill_pikounis at merck.com
Phone: 732 594 3913
Fax: 732 594 1565
-----Original Message-----
From: Rob Lambkin [mailto:r.lambkin at retroscreen.com]
Sent: Thursday, April 17, 2003 4:51 AM
To: r-help at stat.math.ethz.ch
Cc: Shobana Balasingam; Seb Bossuyt; Katie Benjamin; Alex Mann
Subject: [R] Validation of R
Hi All
I am really very interested in starting to use R within our company. I
particularly like the open source nature of the product. My
company is a
medical research company which is part of the University of London.
We conduct contract virology research for large pharma companies. My
question is how do we validate this software? I wonder if anyone else
has had the problem and might be able to comment.
Thanks
Rob
Robert Lambkin BSc (Hon's), MRPharmS, PhD
Director and General Manager
Retroscreen Limited
Retroscreen Virology Limited
The Medical Building, Queen Mary, University of London, 327 Mile End
Road, London, E1 4NS
Tel: 020 7882 7624 Fax: 020 7882 6990
(Retroscreen Virology Ltd. Registered in England & Wales No:2326557)
The information contained in this message is confidential and is
intended for the addressee(s) only. If you have received this
message in
error or there are any problems please notify the originator
immediately.
Unauthorised use, disclosure, copying or alteration of this message is
strictly forbidden.
Retroscreen Virology Limited will not be liable for any
action taken in
reliance on the data contained in this e-mail as it may not have been
quality assessed or assured.