Skip to content

Fisher LDA and prior=c(...) argument

4 messages · Edoardo M Airoldi, Edgar Acuna, Brian Ripley

Edoardo M Airoldi
# 
hello,
 I am using LDA and QDA function of MASS library.  I understand Fisher LDA 
is a method non-probabilistic in nature, so I wonder what happens when I 
try to predict my test set examples as in:
Specifically I ask this because in my problem there are 700 examples 
class A, and 50 in class B, and I'd be glad to use a way to weight the 
contribution of the examples in different classes.
 My guess is that the CODE above estimates the likelihood of of the Fisher 
scores for (example | class) and then implements the Bayes rule to return 
the maximum a-posteriori class.

 Is that correct?  Any pointer towards that direction is appreciated.  
Please cc to edo at stat.cmu.edu your reply.
Thanks

Edoardo M. Airoldi
http://www.stat.cmu.edu/~eairoldi
BH 232L  (412) 268.7829
PC Lab   (412) 268.8719
Edoardo M Airoldi
# 
hello,
 a clarification.

  I am using LDA and QDA function of MASS library.  I understand Fisher
LDA is a method non-probabilistic in nature, so I wonder what happens when
I try to predict my test set examples as in:
Specifically I ask this because in my problem there are 700 examples
class A, and 50 in class B, and I'd be glad to use a way to weight the
contribution of the examples in different classes (in the prediction
stage for LDA I guess)

  My guess is that the CODE above estimates the likelihood of 'the
projection of the data onto the canonical variate' (only one with 2
classes) as in:  P(example | class=.)  and then implements the Bayes
rule to return the maximum a-posteriori class, using the estimated 
likelihood and the given prior=c(...)
  
  Is that correct?  Any pointer towards the understanding is appreciated.

  Further any pointer towards an example that uses the argument CV=TRUE is 
also appreciated, since i was not able (apparently) to get any change by 
setting it to TRUE  =:-)

Edoardo M. Airoldi
http://www.stat.cmu.edu/~eairoldi
BH 232L  (412) 268.7829
PC Lab   (412) 268.8719
Edgar Acuna
# 
Hello,

If you don not use the option prior in lda then the priors  are estimated
proportionally to the size of each class. If you do not use the option
prior in predict.lda then they are taken from the lda object. Otherwise
you can use any numbers in the vector of priors the only condition is that
they add up one.

The CV option of lda refers to the leave-out-out method. Several people
use n-fold CV. I wrote an R function to compute 10-fold CV for lda. Look
at :
math.uprm.edu/~edgar/CV10LDA.TXT

Regards,

Edgar Acuna
On Sun, 18 May 2003, Edoardo M Airoldi wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
MASS the package provides suport for MASS the book.  See the latter for 
the details, also my book

Ripley, B. D. (1996) Pattern Recognition and Neural Networks. CUP

which explains leave-one-out CV and has the formulae.

Someone else mention n-fold CV.  That, including S code and examples, is 
in the MASS book and its scripts are in the MASS package.

Please do not ignore the DESCRIPTION files of a package:
library(help=MASS) says

Description:

Package: MASS
Description: The main library.
Title: Main Library of Venables and Ripley's MASS
...
BundleDescription: Various functions from the software of Venables and
        Ripley, `Modern Applied Statistics with S' (4th edition).

        
On Sun, 18 May 2003, Edoardo M Airoldi wrote:

        

            

      
      
      
    

        
Who mentioned Fisher LDA?  Fisher is not mentioned on the help page, and 
what is usually called LDA is not due to Fisher.