Message-ID: <BE4DC2CA.3C29%sdavis2@mail.nih.gov>
Date: 2005-03-04T12:46:18Z
From: Sean Davis
Subject: Rank-based p-value on large dataset
In-Reply-To: <200503031640.55824.deepayan@stat.wisc.edu>
On 3/3/05 17:40, "Deepayan Sarkar" <deepayan at stat.wisc.edu> wrote:
> On Thursday 03 March 2005 16:32, Deepayan Sarkar wrote:
>> On Thursday 03 March 2005 16:22, Sean Davis wrote:
>>> I have a fairly simple problem--I have about 80,000 values (call
>>> them y) that I am using as an empirical distribution and I want to
>>> find the p-value (never mind the multiple testing issues here, for
>>> the time being) of 130,000 points (call them x) from the empirical
>>> distribution. I typically do that (for one-sided test) something
>>> like
>>>
>>> loop over i in x
>>> p.val[i] = sum(y>x[i])/length(y)
>>>
>>> and repeat for all i. However, length(x) is large here as is
>>> length(y), so this process takes quite a long time. Any
>>> suggestions?
>>
>> The obvious thing to do would be
>>
>> p.val = 1 - ecdf(x)(y)
>
> or rather: p.val = 1 - ecdf(y)(x)
>
Deepayan,
Thanks (and to Martin, also). This works wonderfully. I didn't expect such
a function to exist, but knowing of it will simplify matters significantly
for me.
Sean