subject: Log likelihood above 0

Hi -
 
 In an effort to learn some basic arima modeling in R i went through
 the tutorial found at
 
 
 One of the examples gave me a log likelihood of 77. Now I am simply
 wondering if this is the expected behavior? Looking in my text book
 this should not be possible. I have actually spent some time on this
 but neither the documentation ?arima or google gave me a satisfying
 answer.
 
 
 
 Data and code:
 
 gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45,
 -0.23, 0.04, -0.22, -0.55
 , -0.40,  -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05,
 -0.05, -0.30, -0.35
 , -0.42,  -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33,
 -0.28, -0.02,  0.06
 , -0.20,  -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11,
 -0.15,  0.04, -0.05
 ,  0.01,  -0.22, -0.03,  0.03,  0.04, -0.11,  0.05, -0.08,  0.01,
 0.12,  0.15, -0.02
 ,  0.14,   0.11,  0.10,  0.06,  0.10, -0.01,  0.01,  0.12, -0.03,
 -0.09, -0.17, -0.02
 ,  0.03,   0.12, -0.09, -0.09, -0.18,  0.08,  0.10,  0.05, -0.02,
 0.10,  0.05,  0.03
 , -0.25,  -0.15, -0.07, -0.02, -0.09,  0.00,  0.04, -0.10, -0.05,
 0.18, -0.06, -0.02
 , -0.21,   0.16,  0.07,  0.13,  0.27,  0.40,  0.10,  0.34,  0.16,
 0.13,  0.19,  0.35
 ,  0.42,   0.28,  0.49,  0.44,  0.16,  0.18,  0.31,  0.47,  0.36,
 0.40,  0.71,  0.43
 ,  0.41,   0.56,  0.70,  0.66,  0.60)
 
 gTemp.ts = ts(gTemp.raw, start=1880, freq=1)
 
 gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1))
 
 
 
 Results:
 

 > gTemp.model

 
 Call:
 arima(x = diff(gTemp.ts), order = c(1, 0, 1))
 
 Coefficients:
          ar1      ma1         intercept
        0.2695  -0.8180     0.0061
 s.e.  0.1122   0.0624     0.0030
 
 sigma^2 estimated as 0.01680:  log likelihood = 77.05,  aic = -146.11
 

Likelihood is a function of the parameters, conditioned upon the data.  It is not the same as a probability density function.  Terms or factors which do not involve parameters can be omitted from the likelihood function.  For continuous random variables, the density function can be in (0, Inf).  Therefore, the likelihood function can assume any value between 0 and Inf.  Hence the log-likelihood can be in (-Inf, Inf).  

When the random variable is discrete, the density or probability mass function cannot be greater than 1.   Hence the likelihood cannot be greater than 1, in which case, the log-likelihood cannot be positive.

Ravi.

 On Oct 5, 2010, at 15:36 , Ravi Varadhan wrote:
 

 > Likelihood is a function of the parameters, conditioned upon the 

data.  It is not the same as a probability density function.  Terms or 
factors which do not involve parameters can be omitted from the 
likelihood function.  For continuous random variables, the density 
function can be in (0, Inf).  Therefore, the likelihood function can 
assume any value between 0 and Inf.  Hence the log-likelihood can be 
in (-Inf, Inf).  

 > 
 > When the random variable is discrete, the density or probability 

mass function cannot be greater than 1.   Hence the likelihood cannot 
be greater than 1, in which case, the log-likelihood cannot be positive.
 
 ...unless one of the above mentioned terms that do not involve 
parameters is omitted. E.g. the Poisson likelihood is
 
 x log lambda - lambda - log(x!)
 
 and the sum of the first two terms can easily be positive.
 
 

 > 
 > Ravi.
 > ____________________________________________________________________
 > 
 > Ravi Varadhan, Ph.D.
 > Assistant Professor,
 > Division of Geriatric Medicine and Gerontology
 > School of Medicine
 > Johns Hopkins University
 > 
 > Ph. (410) 502-2619
 > email: rvaradhan at jhmi.edu
 > 
 > 
 > ----- Original Message -----
 > From: Daniel Haugstvedt <daniel.haugstvedt at gmail.com>
 > Date: Tuesday, October 5, 2010 9:16 am
 > Subject: [R] subject: Log likelihood above 0
 > To: r-help at r-project.org
 > 
 > 

 >> Hi -
 >> 
 >> In an effort to learn some basic arima modeling in R i went through
 >> the tutorial found at
 >> 
 >> 
 >> One of the examples gave me a log likelihood of 77. Now I am simply
 >> wondering if this is the expected behavior? Looking in my text book
 >> this should not be possible. I have actually spent some time on this
 >> but neither the documentation ?arima or google gave me a satisfying
 >> answer.
 >> 
 >> 
 >> 
 >> Data and code:
 >> 
 >> gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45,
 >> -0.23, 0.04, -0.22, -0.55
 >> , -0.40,  -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05,
 >> -0.05, -0.30, -0.35
 >> , -0.42,  -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33,
 >> -0.28, -0.02,  0.06
 >> , -0.20,  -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11,
 >> -0.15,  0.04, -0.05
 >> ,  0.01,  -0.22, -0.03,  0.03,  0.04, -0.11,  0.05, -0.08,  0.01,
 >> 0.12,  0.15, -0.02
 >> ,  0.14,   0.11,  0.10,  0.06,  0.10, -0.01,  0.01,  0.12, -0.03,
 >> -0.09, -0.17, -0.02
 >> ,  0.03,   0.12, -0.09, -0.09, -0.18,  0.08,  0.10,  0.05, -0.02,
 >> 0.10,  0.05,  0.03
 >> , -0.25,  -0.15, -0.07, -0.02, -0.09,  0.00,  0.04, -0.10, -0.05,
 >> 0.18, -0.06, -0.02
 >> , -0.21,   0.16,  0.07,  0.13,  0.27,  0.40,  0.10,  0.34,  0.16,
 >> 0.13,  0.19,  0.35
 >> ,  0.42,   0.28,  0.49,  0.44,  0.16,  0.18,  0.31,  0.47,  0.36,
 >> 0.40,  0.71,  0.43
 >> ,  0.41,   0.56,  0.70,  0.66,  0.60)
 >> 
 >> gTemp.ts = ts(gTemp.raw, start=1880, freq=1)
 >> 
 >> gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1))
 >> 
 >> 
 >> 
 >> Results:
 >> 

 >>> gTemp.model

 >> 
 >> Call:
 >> arima(x = diff(gTemp.ts), order = c(1, 0, 1))
 >> 
 >> Coefficients:
 >>          ar1      ma1         intercept
 >>        0.2695  -0.8180     0.0061
 >> s.e.  0.1122   0.0624     0.0030
 >> 
 >> sigma^2 estimated as 0.01680:  log likelihood = 77.05,  aic = -146.11
 >> 
 >> ______________________________________________
 >> R-help at r-project.org mailing list
 >> 
 >> PLEASE do read the posting guide 

 > 
 > ______________________________________________
 > R-help at r-project.org mailing list
 > 
 > PLEASE do read the posting guide 

 
 -- 
 Peter Dalgaard
 Center for Statistics, Copenhagen Business School
 Solbjerg Plads 3, 2000 Frederiksberg, Denmark
 Phone: (+45)38153501
 Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com