Skip to content

Adjusting p values of a matrix

8 messages · January Weiner, Bert Gunter, Peter Dalgaard +2 more

January Weiner
# 
Dear all,

I have an n x n matrix of p-values. The matrix is symmetrical, as it
describes the "each against each" p values of correlation
coefficients.

How can I best correct the p values of the matrix? Notably, the total
number of the tests performed is n(n-1)/2, since I do not test the
correlation of each variable with itself. That means, I only want to
correct one half of the matrix, not including the diagonal. Therefore,
simply writing

pmat <- p.adjust( pmat, method= "fdr" )
# where pmat is an n x n matrix

...doesn't cut it.

Of course, I can turn the matrix in to a three column data frame with
n(n-1)/2 rows, but that is slow and not elegant.

regards,
j.
Bert Gunter
# 
1. This is not an R question, AFAICS.

2. Sounds like a research topic.  I don't think there's a meaningful
simple answer. I suspect it strongly depends on the model and context.

-- Bert

On Mon, Apr 4, 2011 at 8:02 AM, January Weiner
<january.weiner at mpiib-berlin.mpg.de> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Peter Dalgaard
    

    

      
      #
    

  


  

      
On Apr 4, 2011, at 17:02 , January Weiner wrote:

        

            

      
      
      
    

        
I don't think there's a really elegant way (have a look inside pairwise.table if you care).

If you start one step further back, you could just use pairwise.table with a suitably defined comparison function. Otherwise, how about

ltri <- lower.tri(pmat)
utri <- upper.tri(pmat)
pmat[ltri] <- p.adjust(pmat[ltri], method = "fdr")
pmat[utri] <- t(pmat)[utri]

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Spencer Graves
    

    

      
      #
    

  


  

      
There are, however, the multcomp and multcompView packages that 
might provide something of interest in this regard.  "multcomp" has a 
companion book,  "Multiple Comparisons Using R" (Bretz, Hothorn, 
Westfall, 2010, CRC Press), which I believe provides an excellent 
overview of the state of the art in multiple comparisons.  The simple 
rule is Bonferroni, which involves multiplying the p-values by n or 
n(n-1)/2.


       Note, also, that one of the most important innovations in 
statistical methods of the past quarter century is the development of 
"false discovery rate", which estimates the false alarm rate among the 
cases that the user actually sees, which is a mixture of true and false 
hypotheses.  The p value, by contrast, is the probability of a decision 
error only among hypotheses that are true.


       For more info, see the Wikipedia entries on Bonferroni or false 
discovery rate -- or the book by Bretz, Hothorn and Westfall or the 
vignettes accompanying the multcomp package.


       Hope this helps.
       Spencer

        
On 4/4/2011 8:54 AM, Bert Gunter wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Spencer Graves
    

    

      
      #
    

  


  

      
There are also the multcomp and multcompView packages that might 
provide something of interest in this regard.  "multcomp" has a 
companion book,  "Multiple Comparisons Using R" (Bretz, Hothorn, 
Westfall, 2010, CRC Press), which I believe provides an excellent 
overview of the state of the art in multiple comparisons.  The simple 
rule is Bonferroni, which involves multiplying the p-values by n or 
n(n-1)/2.


       Note, also, that one of the most important innovations in 
statistical methods of the past quarter century is the development of 
"false discovery rate", which estimates the false alarm rate among the 
cases that the user actually sees, which is a mixture of true and false 
hypotheses.  The p value, by contrast, is the probability of a decision 
error only among hypotheses that are true.


       For more info, see the Wikipedia entries on Bonferroni or false 
discovery rate -- or the book by Bretz, Hothorn and Westfall or the 
vignettes accompanying the multcomp package.


       Hope this helps.
       Spencer

        
On 4/4/2011 8:54 AM, Bert Gunter wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Benno Pütz
    

    

      
      #
    

  


  

      
How about

	as.matrix(p.adjust(as.dist(pmat)))


Benno

        
On 4.Apr.2011, at 17:02, January Weiner wrote:

        

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      January Weiner
    

    

      
      #
    

  


  

          

      
      
      
    

        
Perfect! Thanks.

j.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      January Weiner
    

    

      
      #
    

  


  

          

      
      
      
    

        
I am afraid I was not clear enough. I am wondering how to best correct
p values that are stored in a matrix, or, in more general: how to
apply a function that takes a vector as an argument to the upper right
(or, equivalently, lower left) half of a matrix, excluding the
diagonal. for... in loop is a trivial, but slow and not elegant
solution.

Naturally, what correction should I use in case of tests which clearly
are not independent is another matter, and I agree on that with you.

Best regards,

January