plm random effect: the estimated variance of the individual effect is negative - R-help

Fri, Jan 4, 2013 7:26 AM #

Matteo,

I fully agree with David: please read the posting guide.

Anyway, the error message says it all: "the estimated variance of the
individual effect is negative". See e.g. the "basic panel" chapter (10
or 11) in Wooldridge's "Econometric Analysis of XS and Panel Data" to
understand why this may happen.  

Stata's behaviour is (as far as I remember) to transparently substitute
it with 0; 'plm's is to let the estimation fail, issuing a warning. You
can try a different 'random.method' which may, or might not, solve the
problem, depending on your data. 

Hint: a random effects model with sigma_mu=0 is a pooling model.

HTH

Giovanni Millo, PhD
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 3,
34132 Trieste (Italy)
tel. +39 040 671184
fax  +39 040 671160

------------ original message ------------------

Message: 49
Date: Thu, 3 Jan 2013 13:23:07 -0800
From: David Winsemius <dwinsemius at comcast.net>
To: matteo ognibene <ognibenematteo at hotmail.it>
Cc: "r-help at r-project.org" <r-help at r-project.org>
Subject: Re: [R] plm random effect: the estimated variance of the
	individual	effect is negative
Message-ID: <D32FDC11-38AE-4AAF-9270-30C791232A24 at comcast.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

On Jan 3, 2013, at 8:19 AM, matteo ognibene wrote:

Head up; matteo: This is pretty much unreadable (due to the failure to  
follow the recommended posting procedure.)

David Winsemius, MD
Alameda, CA, USA



--------------- end original message ---------------

?
Ai sensi del D.Lgs. 196/2003 si precisa che le informazi...{{dropped:12}}

matteo ognibene

Fri, Jan 4, 2013 11:08 AM #

Thanks Giovanni,

but I already tried with the others "random.method" and it doesn't work anyway.
Do you think is it possible change the if statement in the plm formula, to set the value to 0 if sigma<0 ?
Because if I insert a dummy variabile in the formula R gives me this error:

Thanks
Matteo Ognibene


----------------------------------------

Subject: Re: [R] plm random effect: the estimated variance of the individual effect is negative
Date: Fri, 4 Jan 2013 16:26:01 +0100
From: Giovanni_Millo at Generali.com
To: ognibenematteo at hotmail.it
CC: r-help at r-project.org

Matteo,

I fully agree with David: please read the posting guide.

Anyway, the error message says it all: "the estimated variance of the
individual effect is negative". See e.g. the "basic panel" chapter (10
or 11) in Wooldridge's "Econometric Analysis of XS and Panel Data" to
understand why this may happen.

Stata's behaviour is (as far as I remember) to transparently substitute
it with 0; 'plm's is to let the estimation fail, issuing a warning. You
can try a different 'random.method' which may, or might not, solve the
problem, depending on your data.

Hint: a random effects model with sigma_mu=0 is a pooling model.

HTH

Giovanni Millo, PhD
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 3,
34132 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160

------------ original message ------------------

Message: 49
Date: Thu, 3 Jan 2013 13:23:07 -0800
From: David Winsemius <dwinsemius at comcast.net>
To: matteo ognibene <ognibenematteo at hotmail.it>
Cc: "r-help at r-project.org" <r-help at r-project.org>
Subject: Re: [R] plm random effect: the estimated variance of the
individual effect is negative
Message-ID: <D32FDC11-38AE-4AAF-9270-30C791232A24 at comcast.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

On Jan 3, 2013, at 8:19 AM, matteo ognibene wrote:

I have a problem with the variance estimation in a random effect model
I used this formula to get my result, and it works:reg <- deltaF ~
L1.deltaF + L2.deltaF + deltaCDS + L1.deltaCDS + L2.deltaCDS +
L3.deltaCDS + deltaUS_Yields + deltaZ + L1.deltaZ + L2.deltaZfixed
<- plm(reg, data=Mody_R, na.action = na.exclude,
index=c("country_id", "datestata"), model="random")
but when i split my dataframe into several groups it gives me this
error:"Error in swar(object, data, effect) : the estimated
variance of the individual effect is negative"
The code i used to split the dataframe is:SplitMody <- split(Mody_R,
Mody_R$period)SplitFixed <- lapply(SplitMody, function(Mody_R)
plm(reg, data=Mody_R, na.action = na.exclude, index=c("country_id",
"datestata"), model="random"))
I want to point the all the code works perfectly in a within model,
and I tried using all the random.method for the variance
estimation.In addition I used the same model in Stata and it
works... (the results are identical for a within model in R and Stata)
I'll be really gratefull if someone can help me.
Thanks,Matteo

[[alternative HTML version deleted]]

Head up; matteo: This is pretty much unreadable (due to the failure to
follow the recommended posting procedure.)

--
David Winsemius, MD
Alameda, CA, USA



--------------- end original message ---------------


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.

Millo Giovanni

Sat, Jan 5, 2013 1:10 AM #

Dear Matteo,
as I stated in my previous email, if the random effects variance is set to zero then they are identically zero (as mean=0, sd=0) hence you just need to use 'plm(<...> , model="pooling")'.
Best,
Giovanni

-----Messaggio originale-----
Da: matteo ognibene [mailto:ognibenematteo at hotmail.it] 
Inviato: venerd? 4 gennaio 2013 20.08
A: Millo Giovanni; r-help at r-project.org
Oggetto: RE: [R] plm random effect: the estimated variance of the individual effect is negative


Thanks Giovanni,

but I already tried with the others "random.method" and it doesn't work anyway.
Do you think is it possible change the if statement in the plm formula, to set the value to 0 if sigma<0 ?
Because if I insert a dummy variabile in the formula R gives me this error:

Thanks
Matteo Ognibene


----------------------------------------

Subject: Re: [R] plm random effect: the estimated variance of the 
individual effect is negative
Date: Fri, 4 Jan 2013 16:26:01 +0100
From: Giovanni_Millo at Generali.com
To: ognibenematteo at hotmail.it
CC: r-help at r-project.org

Matteo,

I fully agree with David: please read the posting guide.

Anyway, the error message says it all: "the estimated variance of the 
individual effect is negative". See e.g. the "basic panel" chapter (10 
or 11) in Wooldridge's "Econometric Analysis of XS and Panel Data" to 
understand why this may happen.

Stata's behaviour is (as far as I remember) to transparently 
substitute it with 0; 'plm's is to let the estimation fail, issuing a 
warning. You can try a different 'random.method' which may, or might 
not, solve the problem, depending on your data.

Hint: a random effects model with sigma_mu=0 is a pooling model.

HTH

Giovanni Millo, PhD
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 3,
34132 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160

------------ original message ------------------

Message: 49
Date: Thu, 3 Jan 2013 13:23:07 -0800
From: David Winsemius <dwinsemius at comcast.net>
To: matteo ognibene <ognibenematteo at hotmail.it>
Cc: "r-help at r-project.org" <r-help at r-project.org>
Subject: Re: [R] plm random effect: the estimated variance of the 
individual effect is negative
Message-ID: <D32FDC11-38AE-4AAF-9270-30C791232A24 at comcast.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

On Jan 3, 2013, at 8:19 AM, matteo ognibene wrote:

Hi,R 2.15.2 plm() function on Windows 7 I have a problem with the 
variance estimation in a random effect model I used this formula to 
get my result, and it works:reg <- deltaF ~ L1.deltaF + L2.deltaF + 
deltaCDS + L1.deltaCDS + L2.deltaCDS + L3.deltaCDS + deltaUS_Yields 
+ deltaZ + L1.deltaZ + L2.deltaZfixed
<- plm(reg, data=Mody_R, na.action = na.exclude, 
index=c("country_id", "datestata"), model="random") but when i split 
my dataframe into several groups it gives me this error:"Error in 
swar(object, data, effect) : the estimated variance of the 
individual effect is negative"
The code i used to split the dataframe is:SplitMody <- split(Mody_R, 
Mody_R$period)SplitFixed <- lapply(SplitMody, function(Mody_R) 
plm(reg, data=Mody_R, na.action = na.exclude, index=c("country_id", 
"datestata"), model="random")) I want to point the all the code 
works perfectly in a within model, and I tried using all the 
random.method for the variance estimation.In addition I used the 
same model in Stata and it works... (the results are identical for a 
within model in R and Stata) I'll be really gratefull if someone can 
help me.
Thanks,Matteo

[[alternative HTML version deleted]]

Head up; matteo: This is pretty much unreadable (due to the failure to 
follow the recommended posting procedure.)

--
David Winsemius, MD
Alameda, CA, USA



--------------- end original message ---------------


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.