OT: A test with dependent samples.

Ah, experimental units,again ... a subject little taught by 
statisticians
that is often the crux of the matter. As here.

The cat is the experimental unit. There are 73 of them. 12 of them
experienced vomiting after treatment. What's a confidence interval for 
the
true proportion based on our sample of 73? binom.test(12,72) gives us 
.088
to .27 for an exact 2 sided interval (and a P value of 2.2e-16 for the 
null
= 0).

Seems rather convincing -- and simple -- to me!

-- Bert Gunter

-----Original Message-----
From: r-help-bounces at r-project.org 
[mailto:r-help-bounces at r-project.org] On
Behalf Of David Winsemius
Sent: Tuesday, February 10, 2009 1:51 PM
To: Rolf Turner
Cc: R-help Forum
Subject: Re: [R] OT: A test with dependent samples.

In the biomedical arena, at least as I learned from Rosner's 
introductory text, the usual approach to analyzing paired 2 x 2 tables 
is McNemar's test.

?mcnemar.test

mcnemar.test(matrix(c(73,0,61,12),2,2))

	McNemar's Chi-squared test with continuity correction

data:  matrix(c(73, 0, 61, 12), 2, 2)
McNemar's chi-squared = 59.0164, df = 1, p-value = 1.564e-14

The help page has citation to Agresti.

-- 
David winsemius
On Feb 10, 2009, at 4:33 PM, Rolf Turner wrote:

I am appealing to the general collective wisdom of this
list in respect of a statistics (rather than R) question.  This 
question
comes to me from a friend who is a veterinary oncologist.  In a 
study that
she is writing up there were 73 cats who were treated with a drug 
called
piroxicam.  None of the cats were observed to be subject to vomiting 
prior
to treatment; 12 of the cats were subject to vomiting after treatment
commenced.  She wants to be able to say that the treatment had a 
``significant''
impact with respect to this unwanted side-effect.

Initially she did a chi-squared test.  (Presumably on the matrix
matrix(c(73,0,61,12),2,2) --- she didn't give details and I didn't 
pursue
this.) I pointed out to her that because of the dependence --- same 
73
cats pre- and post- treatment --- the chi-squared test is 
inappropriate.

So what *is* appropriate?  There is a dependence structure of some 
sort,
but it seems to me to be impossible to estimate.

After mulling it over for a long while (I'm slow!) I decided that a
non-parametric approach, along the following lines, makes sense:

We have 73 independent pairs of outcomes (a,b) where a or b is 0
if the cat didn't barf, and is 1 if it did barf.

We actually observe 61 (0,0) pairs and 12 (0,1) pairs.

If there is no effect from the piroxicam, then (0,1) and (1,0) are
equally likely.  So given that the outcome is in {(0,1),(1,0)} the
probability of each is 1/2.

Thus we have a sequence of 12 (0,1)-s where (under the null 
hypothesis)
the probability of each entry is 1/2.  Hence the probability of this
sequence is (1/2)^12 = 0.00024.  So the p-value of the (one-sided) 
test
is 0.00024.  Hence the result is ``significant'' at the usual levels,
and my vet friend is happy.

I would very much appreciate comments on my reasoning.  Have I made 
any
goof-ups, missed any obvious pit-falls?  Gone down a wrong garden 
path?

Is there a better approach?

Most importantly (!!!): Is there any literature in which this 
approach is
spelled out?  (The journal in which she wishes to publish will 
almost surely
demand a citation.  They *won't* want to see the reasoning spelled 
out in
the paper.)

I would conjecture that this sort of scenario must arise reasonably 
often
in medical statistics and the suggested approach (if it is indeed 
valid
and sensible) would be ``standard''.  It might even have a name! 
But I
have no idea where to start looking, so I thought I'd ask this 
wonderfully
learned list.

Thanks for any input.

	cheers,

		Rolf Turner


######################################################################
Attention:\ This e-mail message is privileged and confid...{{dropped: 
9}}

The only question at issue (i.e. capable of being addressed) is: is  
giving
the drug to non-vomiting cats associated with vomiting? (I would  
strongly
suspect that cats that were vomiting beforehand would have been  
excluded
from the study, as the researcher would have felt that one couldn't  
then
tell whether or not the drug caused vomiting problems for them. No?)

There were 73 non-vomiting cats, 12 of which started vomiting after
receiving the drug. All I can do is give a confidence interval for the
estimated proportion of nonvomiting cats that vomit when given this  
drug and
perhaps ask whether it is consistent with their nonvomiting status  
before.
Which is what I did. And it's pretty convincing that giving the  
pill is
associated with vomiting, right?

Whether the vomiting was associated with the giving of this  
**particular**
drug is, of course, impossible to tell, because the researcher  
failed to
include placebo controls. I chose 0 for a null as a representation  
of their
non-vomiting status, but the scientific question of interest is  
probably to
compare them to the proportion of cats that would vomit if given  
any pill at
all. Without any placebo controls, who can tell? Substitute a prior  
guess if
you like for a Null. Which is exactly the point that Marc Schwartz  
made --
that is, that the data are probably completely useless to answer the
question of interest because the researcher messed up the design.

On 11/02/2009, at 1:06 PM, Bert Gunter wrote:

The only question at issue (i.e. capable of being addressed) is: is giving
the drug to non-vomiting cats associated with vomiting? (I would strongly
suspect that cats that were vomiting beforehand would have been excluded
from the study, as the researcher would have felt that one couldn't then
tell whether or not the drug caused vomiting problems for them. No?)

There were 73 non-vomiting cats, 12 of which started vomiting after
receiving the drug. All I can do is give a confidence interval for the
estimated proportion of nonvomiting cats that vomit when given this drug
and
perhaps ask whether it is consistent with their nonvomiting status before.
Which is what I did. And it's pretty convincing that giving the pill is
associated with vomiting, right?

Whether the vomiting was associated with the giving of this **particular**
drug is, of course, impossible to tell, because the researcher failed to
include placebo controls. I chose 0 for a null as a representation of
their
non-vomiting status, but the scientific question of interest is probably
to
compare them to the proportion of cats that would vomit if given any pill
at
all. Without any placebo controls, who can tell? Substitute a prior guess
if
you like for a Null. Which is exactly the point that Marc Schwartz made --
that is, that the data are probably completely useless to answer the
question of interest because the researcher messed up the design.

I appreciate the time and trouble that several people have taken to attempt
to answer my somewhat inchoate question.  I'm still trying to get my head
around McNemar's test, plus other ideas and suggestions.  As I've said,
I'm slow.  I am also remiss in never before having come to grips with
McNemar
before now. Just another of the many lacunae in my knowledge.

I would like however to clarify (I hope) a few points in respect of Bert's
comments above.

The study was a retrospective study on cats being treated for cancer.
The objective, as I understand it, was basically to consider unwanted
side effects of the drug piroxicam which was used (in combination with
other therapies) on all 73 cats.

So it's not really correct to say that cats who were vomiting before
hand would've been excluded from the study.  The cats were being treated
for cancer, not studied.  The study came afterward.  It is possible that
a vet might say ``Oh-oh; this cat's been vomiting.  So we shouldn't use
the piroxicam treatment, we should do something else.''  It's possible,
but I would guess not.  I believe that piroxicam is a relatively new
treatment, and its side effects are still being figured out.

Cats do generally have a tendency to vomit from time to time.  A null
hypothesis of p = 0 is unrealistic, and moreover *any* incident of vomiting
would irrefutably disprove p = 0, wouldn't it?  It would be nice to have
some realistic value of p_0 for the probability of vomiting under ``normal''
circumstances, but that's just not available.

Placebo controls could not possibly come into the situation at all.  This
was real life, not a study, and the cats had cancer and had to be treated.
I'm sorry if my original posting was unclear or misleading in this regard.

Finally I think it's unfair to say that my friend ``messed up the design''.
There was no design.  It was a retrospective observational study of
real-life
treatment procedures and their outcomes.

It seems pretty certain to me that the data are not ``completely useless''
to answer the question posed.  There *is* information content there.  The
information may not be ideal, but there is information to be had.  In
particular it seems to me that my ad hoc proposal in my original email
does indeed (``validly'') test the null hypothesis that the treatment
has no effect on the propensity of cats to vomit.

(No one has commented on my proposed test as such; would anyone care to?)

I'm still not sure about McNemar, but I suspect it is ``valid'' as well,
possibly
modulo an assumption about normality and possibly with some other
assumption(s)
being thrown in.


       cheers,

               Rolf Turner

######################################################################
Attention:\ This e-mail message is privileged and confid...{{dropped:9}}