Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore. The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong. However, the corresponding Q-Statistics of the rugarch output are correct! Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts): *---------------------------------* *????????? GARCH Model Fit* *---------------------------------* Conditional Variance Dynamics ??? ----------------------------------- GARCH Model??? : sGARCH(1,1) Mean Model??? : ARFIMA(5,0,5) Distribution??? : norm Optimal Parameters ------------------------------------ ??????? Estimate? Std. Error? t value Pr(>|t|) ar1???? 0.000000????????? NA?????? NA?????? NA ar2???? 0.000000????????? NA?????? NA?????? NA ar3???? 0.000000????????? NA?????? NA?????? NA ar4??? -0.292207??? 0.019550 -14.9467? 0.0e+00 ar5??? -0.745887??? 0.018488 -40.3436? 0.0e+00 ma1???? 0.000000????????? NA?????? NA?????? NA ma2???? 0.000000????????? NA?????? NA?????? NA ma3???? 0.000000????????? NA?????? NA?????? NA ma4???? 0.309446??? 0.026659? 11.6073? 0.0e+00 ma5???? 0.718856??? 0.021208? 33.8952? 0.0e+00 omega?? 0.000006??? 0.000001?? 4.2106? 2.5e-05 alpha1? 0.093397??? 0.011308?? 8.2591? 0.0e+00 beta1?? 0.892404??? 0.012437? 71.7563? 0.0e+00 Q-Statistics on Standardized Residuals ------------------------------------ ???????????????????????? statistic? ? p-value Lag[1]????????? ? ?? ? 7.898???? ? 4.949e-03 Lag[p+q+1][11]??? 21.627???? 3.312e-06 Lag[p+q+5][15]??? 27.133???? 5.374e-05 d.o.f=10 H0 : No serial correlation Q-Statistics on Standardized Squared Residuals ------------------------------------ ???????????? ? ? ? ? ?? statistic? p-value Lag[1]??????? ? ? ?? 1.274???? 0.258961 Lag[p+q+1][3]???? 9.351??? 0.002229 Lag[p+q+5][7]??? 12.135 ?? 0.032980 d.o.f=2 As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant. This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong? One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason? Thanks a lot for your notice. My code: library(rugarch) modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modgarch<-ugarchfit(spec=modsp,data=mydata) plot(modgarch) residuals(mydata,standardize=TRUE) resdi<-as.numeric(residuals(mydata,standardize=TRUE)) library(forecast) Acf(resdi^2) Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2) Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2) -------------- next part -------------- A non-text attachment was scrubbed... Name: mydatagarch.RData Type: application/octet-stream Size: 27552 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/78795877/attachment.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: rugarchoutput.PNG Type: image/png Size: 17372 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/78795877/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: forecastoutput.PNG Type: image/png Size: 10868 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/78795877/attachment-0001.png>
Error in rugarch ACF squared standardized residuals plot
9 messages · Jen Bohold, Alexios Ghalanos, Peter Allington +1 more
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although
you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore.
The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong.
However, the corresponding Q-Statistics of the rugarch output are correct!
Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts):
*---------------------------------*
* GARCH Model Fit*
*---------------------------------*
Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model : ARFIMA(5,0,5)
Distribution : norm
Optimal Parameters
------------------------------------
Estimate Std. Error t value Pr(>|t|)
ar1 0.000000 NA NA NA
ar2 0.000000 NA NA NA
ar3 0.000000 NA NA NA
ar4 -0.292207 0.019550 -14.9467 0.0e+00
ar5 -0.745887 0.018488 -40.3436 0.0e+00
ma1 0.000000 NA NA NA
ma2 0.000000 NA NA NA
ma3 0.000000 NA NA NA
ma4 0.309446 0.026659 11.6073 0.0e+00
ma5 0.718856 0.021208 33.8952 0.0e+00
omega 0.000006 0.000001 4.2106 2.5e-05
alpha1 0.093397 0.011308 8.2591 0.0e+00
beta1 0.892404 0.012437 71.7563 0.0e+00
Q-Statistics on Standardized Residuals
------------------------------------
statistic p-value
Lag[1] 7.898 4.949e-03
Lag[p+q+1][11] 21.627 3.312e-06
Lag[p+q+5][15] 27.133 5.374e-05
d.o.f=10
H0 : No serial correlation
Q-Statistics on Standardized Squared Residuals
------------------------------------
statistic p-value
Lag[1] 1.274 0.258961
Lag[p+q+1][3] 9.351 0.002229
Lag[p+q+5][7] 12.135 0.032980
d.o.f=2
As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant.
This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong?
One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason?
Thanks a lot for your notice.
My code:
library(rugarch)
modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)),
mean.model = list(armaOrder = c(5, 5), include.mean = FALSE),
distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0))
modgarch<-ugarchfit(spec=modsp,data=mydata)
plot(modgarch)
residuals(mydata,standardize=TRUE)
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
library(forecast)
Acf(resdi^2)
Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2)
Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore. The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong. However, the corresponding Q-Statistics of the rugarch output are correct! Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts): *---------------------------------* *? ? ? ? ? GARCH Model Fit* *---------------------------------* Conditional Variance Dynamics ----------------------------------- GARCH Model? ? : sGARCH(1,1) Mean Model? ? : ARFIMA(5,0,5) Distribution? ? : norm Optimal Parameters ------------------------------------ ? ? ? ? ? Estimate? Std. Error? t value Pr(>|t|) ar1? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar2? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar3? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar4? ? -0.292207? ? 0.019550 -14.9467? 0.0e+00 ar5? ? -0.745887? ? 0.018488 -40.3436? 0.0e+00 ma1? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma2? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma3? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma4? ? 0.309446? ? 0.026659? 11.6073? 0.0e+00 ma5? ? 0.718856? ? 0.021208? 33.8952? 0.0e+00 omega? 0.000006? ? 0.000001? 4.2106? 2.5e-05 alpha1? 0.093397? ? 0.011308? 8.2591? 0.0e+00 beta1? 0.892404? ? 0.012437? 71.7563? 0.0e+00 Q-Statistics on Standardized Residuals ------------------------------------ ? ? ? ? ? ? ? ? ? ? ? ? ? statistic? ? p-value Lag[1]? ? ? ? ? ? ? ? 7.898? ? ? 4.949e-03 Lag[p+q+1][11]? ? 21.627? ? 3.312e-06 Lag[p+q+5][15]? ? 27.133? ? 5.374e-05 d.o.f=10 H0 : No serial correlation Q-Statistics on Standardized Squared Residuals ------------------------------------ ? ? ? ? ? ? ? ? ? ? ? ? ? statistic? p-value Lag[1]? ? ? ? ? ? ? 1.274? ? 0.258961 Lag[p+q+1][3]? ? 9.351? ? 0.002229 Lag[p+q+5][7]? ? 12.135? ? 0.032980 d.o.f=2 As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant. This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong? One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason? Thanks a lot for your notice. My code: library(rugarch) modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modgarch<-ugarchfit(spec=modsp,data=mydata) plot(modgarch) residuals(mydata,standardize=TRUE) resdi<-as.numeric(residuals(mydata,standardize=TRUE)) library(forecast) Acf(resdi^2) Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2) Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
-------------- next part -------------- A non-text attachment was scrubbed... Name: rugarchoutput.PNG Type: image/png Size: 17314 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/87b81ea1/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: forecastoutput.PNG Type: image/png Size: 11031 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/87b81ea1/attachment-0001.png>
I can't replicate your PNG chart differences and here is an example of how you can check: ########################################## library(rugarch) # you specification with YOUR estimated parameters: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0, ar4 = -0.292207, ar5 = -0.745887, ma1=0,ma2=0,ma3=0, ma4 = 0.309446, ma5 = 0.718856, omega = 6e-6, alpha1=0.093397, beta1 = 0.892404)) # Simulate a path sim=ugarchpath(modsp, n.sim=5000) # extract the simulated data mydata = as.numeric(fitted(sim)) # restate the specification: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) # estimate the model: modgarch<-ugarchfit(spec=modsp,data=mydata ) # extract the standardized residuals: resdi<-as.numeric(residuals(modgarch,standardize=TRUE)) # plot: library(forecast) par(mfrow=c(2,2)) Acf(resdi^2) plot(modgarch, which=11) ############################################### I don't see ANY differences. You are however welcome to look at the underlying code in the rugarch-plots.R file in the source. If you find a bug you are welcome to submit a patch to the google code repository of the package rather than continuously sending this list PNG attachments and non reproducible code/examples. Regards, Alexios
On 22/05/2013 11:49, Jen Bohold wrote:
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although
you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore.
The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong.
However, the corresponding Q-Statistics of the rugarch output are correct!
Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts):
*---------------------------------*
* GARCH Model Fit*
*---------------------------------*
Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model : ARFIMA(5,0,5)
Distribution : norm
Optimal Parameters
------------------------------------
Estimate Std. Error t value Pr(>|t|)
ar1 0.000000 NA NA NA
ar2 0.000000 NA NA NA
ar3 0.000000 NA NA NA
ar4 -0.292207 0.019550 -14.9467 0.0e+00
ar5 -0.745887 0.018488 -40.3436 0.0e+00
ma1 0.000000 NA NA NA
ma2 0.000000 NA NA NA
ma3 0.000000 NA NA NA
ma4 0.309446 0.026659 11.6073 0.0e+00
ma5 0.718856 0.021208 33.8952 0.0e+00
omega 0.000006 0.000001 4.2106 2.5e-05
alpha1 0.093397 0.011308 8.2591 0.0e+00
beta1 0.892404 0.012437 71.7563 0.0e+00
Q-Statistics on Standardized Residuals
------------------------------------
statistic p-value
Lag[1] 7.898 4.949e-03
Lag[p+q+1][11] 21.627 3.312e-06
Lag[p+q+5][15] 27.133 5.374e-05
d.o.f=10
H0 : No serial correlation
Q-Statistics on Standardized Squared Residuals
------------------------------------
statistic p-value
Lag[1] 1.274 0.258961
Lag[p+q+1][3] 9.351 0.002229
Lag[p+q+5][7] 12.135 0.032980
d.o.f=2
As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant.
This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong?
One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason?
Thanks a lot for your notice.
My code:
library(rugarch)
modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)),
mean.model = list(armaOrder = c(5, 5), include.mean = FALSE),
distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0))
modgarch<-ugarchfit(spec=modsp,data=mydata)
plot(modgarch)
residuals(mydata,standardize=TRUE)
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
library(forecast)
Acf(resdi^2)
Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2)
Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
It is not true, that I sent non-reproducible code/examples. I can give you the complete code again and I attach my data. You just have to run the code and you will see, that indeed both plots do differ. So this is replicatable. library(rugarch) myspecification<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modelgarch<-ugarchfit(spec=myspecification,data=mydata) plot(modelgarch,which=11) library(forecast) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi) You can see, that the plots are NOT the same. I do not know, what is wrong with your code and I do not want to offend you, but there is clearly an error in it. ----- Urspr?ngliche Message ----- Von: alexios ghalanos <alexios at 4dscape.com> An: Jen Bohold <jenbohold at yahoo.de> CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org> Gesendet: 13:30 Mittwoch, 22.Mai 2013 Betreff: Re: Error in rugarch ACF squared standardized residuals plot I can't replicate your PNG chart differences and here is an example of how you can check: ########################################## library(rugarch) # you specification with YOUR estimated parameters: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0, ar4 = -0.292207, ar5 = -0.745887, ma1=0,ma2=0,ma3=0, ma4 = 0.309446, ma5 = 0.718856, omega = 6e-6, alpha1=0.093397, beta1 = 0.892404)) # Simulate a path sim=ugarchpath(modsp, n.sim=5000) # extract the simulated data mydata = as.numeric(fitted(sim)) # restate the specification: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) # estimate the model: modgarch<-ugarchfit(spec=modsp,data=mydata ) # extract the standardized residuals: resdi<-as.numeric(residuals(modgarch,standardize=TRUE)) # plot: library(forecast) par(mfrow=c(2,2)) Acf(resdi^2) plot(modgarch, which=11) ############################################### I don't see ANY differences. You are however welcome to look at the underlying code in the rugarch-plots.R file in the source. If you find a bug you are welcome to submit a patch to the google code repository of the package rather than continuously sending this list PNG attachments and non reproducible code/examples. Regards, Alexios
On 22/05/2013 11:49, Jen Bohold wrote:
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore. The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong. However, the corresponding Q-Statistics of the rugarch output are correct! Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts): *---------------------------------* *? ? ? ? ? GARCH Model Fit* *---------------------------------* Conditional Variance Dynamics ----------------------------------- GARCH Model? ? : sGARCH(1,1) Mean Model? ? : ARFIMA(5,0,5) Distribution? ? : norm Optimal Parameters ------------------------------------ ? ? ? ? ? ? Estimate? Std. Error? t value Pr(>|t|) ar1? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar2? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar3? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ar4? ? -0.292207? ? 0.019550 -14.9467? 0.0e+00 ar5? ? -0.745887? ? 0.018488 -40.3436? 0.0e+00 ma1? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma2? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma3? ? 0.000000? ? ? ? ? NA? ? ? NA? ? ? NA ma4? ? 0.309446? ? 0.026659? 11.6073? 0.0e+00 ma5? ? 0.718856? ? 0.021208? 33.8952? 0.0e+00 omega? 0.000006? ? 0.000001? 4.2106? 2.5e-05 alpha1? 0.093397? ? 0.011308? 8.2591? 0.0e+00 beta1? 0.892404? ? 0.012437? 71.7563? 0.0e+00 Q-Statistics on Standardized Residuals ------------------------------------ ? ? ? ? ? ? ? ? ? ? ? ? ? ? statistic? ? p-value Lag[1]? ? ? ? ? ? ? ? 7.898? ? ? 4.949e-03 Lag[p+q+1][11]? ? 21.627? ? 3.312e-06 Lag[p+q+5][15]? ? 27.133? ? 5.374e-05 d.o.f=10 H0 : No serial correlation Q-Statistics on Standardized Squared Residuals ------------------------------------ ? ? ? ? ? ? ? ? ? ? ? ? ? ? statistic? p-value Lag[1]? ? ? ? ? ? ? 1.274? ? 0.258961 Lag[p+q+1][3]? ? 9.351? ? 0.002229 Lag[p+q+5][7]? ? 12.135? ? 0.032980 d.o.f=2 As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant. This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong? One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason? Thanks a lot for your notice. My code: library(rugarch) modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modgarch<-ugarchfit(spec=modsp,data=mydata) plot(modgarch) residuals(mydata,standardize=TRUE) resdi<-as.numeric(residuals(mydata,standardize=TRUE)) library(forecast) Acf(resdi^2) Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2) Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
-------------- next part -------------- A non-text attachment was scrubbed... Name: mydatagarch.RData Type: application/octet-stream Size: 27552 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20130522/6a12563e/attachment.obj>
I've traced the difference to the way the residuals method deals with the startup values (5 lags) versus the plot which extracts directly the standardized residuals used in the likelihood routine. This gives the same answer: par(mfrow=c(2,2)) plot(modelgarch,which=11) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi[-c(1:5)]) In most cases this is not likely be significantly different. It is related to a problem of how to initialize the ARMA recursion and what to return to the user. In THIS case, the first 5 values had an effect which changed the visual result marginally. For consistency I guess that the plot should return the result one would expect by using the residuals method on the model. Feel free to submit a patch. Regards, Alexios
On 22/05/2013 13:40, Jen Bohold wrote:
It is not true, that I sent non-reproducible code/examples. I can give you the complete code again and I attach my data. You just have to run the code and you will see, that indeed both plots do differ. So this is replicatable. library(rugarch) myspecification<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modelgarch<-ugarchfit(spec=myspecification,data=mydata) plot(modelgarch,which=11) library(forecast) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi) You can see, that the plots are NOT the same. I do not know, what is wrong with your code and I do not want to offend you, but there is clearly an error in it. ----- Urspr?ngliche Message ----- Von: alexios ghalanos <alexios at 4dscape.com> An: Jen Bohold <jenbohold at yahoo.de> CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org> Gesendet: 13:30 Mittwoch, 22.Mai 2013 Betreff: Re: Error in rugarch ACF squared standardized residuals plot I can't replicate your PNG chart differences and here is an example of how you can check: ########################################## library(rugarch) # you specification with YOUR estimated parameters: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0, ar4 = -0.292207, ar5 = -0.745887, ma1=0,ma2=0,ma3=0, ma4 = 0.309446, ma5 = 0.718856, omega = 6e-6, alpha1=0.093397, beta1 = 0.892404)) # Simulate a path sim=ugarchpath(modsp, n.sim=5000) # extract the simulated data mydata = as.numeric(fitted(sim)) # restate the specification: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) # estimate the model: modgarch<-ugarchfit(spec=modsp,data=mydata ) # extract the standardized residuals: resdi<-as.numeric(residuals(modgarch,standardize=TRUE)) # plot: library(forecast) par(mfrow=c(2,2)) Acf(resdi^2) plot(modgarch, which=11) ############################################### I don't see ANY differences. You are however welcome to look at the underlying code in the rugarch-plots.R file in the source. If you find a bug you are welcome to submit a patch to the google code repository of the package rather than continuously sending this list PNG attachments and non reproducible code/examples. Regards, Alexios On 22/05/2013 11:49, Jen Bohold wrote:
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to comment on me, I thought I should share this to the list, maybe someone else is some time wondering about this (maybe I did a mistake, but no one of the list or you told me in the previous mail). Also, I do not want to offend you, I like your package it's great! Especially I liked the acf plots, they have a better design, although
you will see in the following text, that the "ACF of Squared Standadrized Residuals" plot are not useable anymore.
The plot of the ACF of the squared standardized residuals in rugarch output (you get it via plot(yourmodel) and choosing number 11) is wrong.
However, the corresponding Q-Statistics of the rugarch output are correct!
Consider the following (I attached my data and the plots). I fitted the following model (output extracted to the relevant parts):
*---------------------------------*
* GARCH Model Fit*
*---------------------------------*
Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model : ARFIMA(5,0,5)
Distribution : norm
Optimal Parameters
------------------------------------
Estimate Std. Error t value Pr(>|t|)
ar1 0.000000 NA NA NA
ar2 0.000000 NA NA NA
ar3 0.000000 NA NA NA
ar4 -0.292207 0.019550 -14.9467 0.0e+00
ar5 -0.745887 0.018488 -40.3436 0.0e+00
ma1 0.000000 NA NA NA
ma2 0.000000 NA NA NA
ma3 0.000000 NA NA NA
ma4 0.309446 0.026659 11.6073 0.0e+00
ma5 0.718856 0.021208 33.8952 0.0e+00
omega 0.000006 0.000001 4.2106 2.5e-05
alpha1 0.093397 0.011308 8.2591 0.0e+00
beta1 0.892404 0.012437 71.7563 0.0e+00
Q-Statistics on Standardized Residuals
------------------------------------
statistic p-value
Lag[1] 7.898 4.949e-03
Lag[p+q+1][11] 21.627 3.312e-06
Lag[p+q+5][15] 27.133 5.374e-05
d.o.f=10
H0 : No serial correlation
Q-Statistics on Standardized Squared Residuals
------------------------------------
statistic p-value
Lag[1] 1.274 0.258961
Lag[p+q+1][3] 9.351 0.002229
Lag[p+q+5][7] 12.135 0.032980
d.o.f=2
As you can see in the "Q-Statistics on Standardized Squared Residuals" there is clearly correlation in the standardized squared residuals. BUT if you look at the plot with the plot method and choosing number 11 you can see, that NO spike is significant.
This plot is not correct, I controlled it via the Acf plot of the forecast package and clearly, the spikes are larger! So the second spike is now significant. I control the calculations via the Box.test method using d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch package are correct! So the p-values are indeed 0.002229 and 0.032980. So why is the plot of the rugarch package wrong?
One further notice: In a previous mail, I asked, why the lags in the Q-Statistics on Standardized Squared Residuals are different to the lags used in Q-Statistics on Standardized Residuals. Of course, I have now seen, that the second uses the GARCH parameters, so it is clear, that this has to be equal to two (1+1). I also have to say, that I think, that the ACF of observations plot e.g. is indeed correct (number 4), so it seems, that the plot number 11 uses different scaled residuals? Maybe it uses the non-standardized squared residuals? Could that be the reason?
Thanks a lot for your notice.
My code:
library(rugarch)
modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)),
mean.model = list(armaOrder = c(5, 5), include.mean = FALSE),
distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0))
modgarch<-ugarchfit(spec=modsp,data=mydata)
plot(modgarch)
residuals(mydata,standardize=TRUE)
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
library(forecast)
Acf(resdi^2)
Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2)
Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
Thanks a lot for not giving me up! I have one last question: I wondered about, because in my example the Q-Statistics was really "strong". So the p-values for the lag 3 and 7 were really small (0.002229 and 0.032980). So this indicates, that there is jointly autocorrelation, but the plot showes, that there is just one spike which is slightly significant. Why does this has such a strong impact on the Q-Statistics? The spike is slightly different (it crosses the dashed line) and the p-value of the Q-Statistics is already so small? Why? The other spikes of lag 4 and 5 and so one are not significant? Why is the p-value of the joint test so small? I would have expect that there are lots of significant spikes? Thanks a lot for your wisdom! 2013/5/22 alexios ghalanos <alexios at 4dscape.com>:
I've traced the difference to the way the residuals method deals with the startup values (5 lags) versus the plot which extracts directly the standardized residuals used in the likelihood routine. This gives the same answer: par(mfrow=c(2,2)) plot(modelgarch,which=11) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi[-c(1:5)]) In most cases this is not likely be significantly different. It is related to a problem of how to initialize the ARMA recursion and what to return to the user. In THIS case, the first 5 values had an effect which changed the visual result marginally. For consistency I guess that the plot should return the result one would expect by using the residuals method on the model. Feel free to submit a patch. Regards, Alexios On 22/05/2013 13:40, Jen Bohold wrote:
It is not true, that I sent non-reproducible code/examples. I can give you the complete code again and I attach my data. You just have to run the code and you will see, that indeed both plots do differ. So this is replicatable. library(rugarch) myspecification<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modelgarch<-ugarchfit(spec=myspecification,data=mydata) plot(modelgarch,which=11) library(forecast) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi) You can see, that the plots are NOT the same. I do not know, what is wrong with your code and I do not want to offend you, but there is clearly an error in it. ----- Urspr?ngliche Message ----- Von: alexios ghalanos <alexios at 4dscape.com> An: Jen Bohold <jenbohold at yahoo.de> CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org> Gesendet: 13:30 Mittwoch, 22.Mai 2013 Betreff: Re: Error in rugarch ACF squared standardized residuals plot I can't replicate your PNG chart differences and here is an example of how you can check: ########################################## library(rugarch) # you specification with YOUR estimated parameters: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0, ar4 = -0.292207, ar5 = -0.745887, ma1=0,ma2=0,ma3=0, ma4 = 0.309446, ma5 = 0.718856, omega = 6e-6, alpha1=0.093397, beta1 = 0.892404)) # Simulate a path sim=ugarchpath(modsp, n.sim=5000) # extract the simulated data mydata = as.numeric(fitted(sim)) # restate the specification: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) # estimate the model: modgarch<-ugarchfit(spec=modsp,data=mydata ) # extract the standardized residuals: resdi<-as.numeric(residuals(modgarch,standardize=TRUE)) # plot: library(forecast) par(mfrow=c(2,2)) Acf(resdi^2) plot(modgarch, which=11) ############################################### I don't see ANY differences. You are however welcome to look at the underlying code in the rugarch-plots.R file in the source. If you find a bug you are welcome to submit a patch to the google code repository of the package rather than continuously sending this list PNG attachments and non reproducible code/examples. Regards, Alexios On 22/05/2013 11:49, Jen Bohold wrote:
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to
comment on me, I thought I should share this to the list, maybe someone else
is some time wondering about this (maybe I did a mistake, but no one of the
list or you told me in the previous mail). Also, I do not want to offend
you, I like your package it's great! Especially I liked the acf plots, they
have a better design, although
you will see in the following text, that the "ACF of Squared
Standadrized Residuals" plot are not useable anymore.
The plot of the ACF of the squared standardized residuals in rugarch
output (you get it via plot(yourmodel) and choosing number 11) is wrong.
However, the corresponding Q-Statistics of the rugarch output are
correct!
Consider the following (I attached my data and the plots). I fitted the
following model (output extracted to the relevant parts):
*---------------------------------*
* GARCH Model Fit*
*---------------------------------*
Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model : ARFIMA(5,0,5)
Distribution : norm
Optimal Parameters
------------------------------------
Estimate Std. Error t value Pr(>|t|)
ar1 0.000000 NA NA NA
ar2 0.000000 NA NA NA
ar3 0.000000 NA NA NA
ar4 -0.292207 0.019550 -14.9467 0.0e+00
ar5 -0.745887 0.018488 -40.3436 0.0e+00
ma1 0.000000 NA NA NA
ma2 0.000000 NA NA NA
ma3 0.000000 NA NA NA
ma4 0.309446 0.026659 11.6073 0.0e+00
ma5 0.718856 0.021208 33.8952 0.0e+00
omega 0.000006 0.000001 4.2106 2.5e-05
alpha1 0.093397 0.011308 8.2591 0.0e+00
beta1 0.892404 0.012437 71.7563 0.0e+00
Q-Statistics on Standardized Residuals
------------------------------------
statistic p-value
Lag[1] 7.898 4.949e-03
Lag[p+q+1][11] 21.627 3.312e-06
Lag[p+q+5][15] 27.133 5.374e-05
d.o.f=10
H0 : No serial correlation
Q-Statistics on Standardized Squared Residuals
------------------------------------
statistic p-value
Lag[1] 1.274 0.258961
Lag[p+q+1][3] 9.351 0.002229
Lag[p+q+5][7] 12.135 0.032980
d.o.f=2
As you can see in the "Q-Statistics on Standardized Squared Residuals"
there is clearly correlation in the standardized squared residuals. BUT if
you look at the plot with the plot method and choosing number 11 you can
see, that NO spike is significant.
This plot is not correct, I controlled it via the Acf plot of the
forecast package and clearly, the spikes are larger! So the second spike is
now significant. I control the calculations via the Box.test method using
d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch
package are correct! So the p-values are indeed 0.002229 and 0.032980. So
why is the plot of the rugarch package wrong?
One further notice: In a previous mail, I asked, why the lags in the
Q-Statistics on Standardized Squared Residuals are different to the lags
used in Q-Statistics on Standardized Residuals. Of course, I have now seen,
that the second uses the GARCH parameters, so it is clear, that this has to
be equal to two (1+1). I also have to say, that I think, that the ACF of
observations plot e.g. is indeed correct (number 4), so it seems, that the
plot number 11 uses different scaled residuals? Maybe it uses the
non-standardized squared residuals? Could that be the reason?
Thanks a lot for your notice.
My code:
library(rugarch)
modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder =
c(1, 1)),
mean.model = list(armaOrder = c(5, 5), include.mean = FALSE),
distribution.model =
"norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0))
modgarch<-ugarchfit(spec=modsp,data=mydata)
plot(modgarch)
residuals(mydata,standardize=TRUE)
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
library(forecast)
Acf(resdi^2)
Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2)
Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Peter Allington, M. Sc. CF&RM University of Washington
do not get confused: I answered with my friends account. 2013/5/22 Peter Allington <peterallington03 at gmail.com>:
Thanks a lot for not giving me up! I have one last question: I wondered about, because in my example the Q-Statistics was really "strong". So the p-values for the lag 3 and 7 were really small (0.002229 and 0.032980). So this indicates, that there is jointly autocorrelation, but the plot showes, that there is just one spike which is slightly significant. Why does this has such a strong impact on the Q-Statistics? The spike is slightly different (it crosses the dashed line) and the p-value of the Q-Statistics is already so small? Why? The other spikes of lag 4 and 5 and so one are not significant? Why is the p-value of the joint test so small? I would have expect that there are lots of significant spikes? Thanks a lot for your wisdom! 2013/5/22 alexios ghalanos <alexios at 4dscape.com>:
I've traced the difference to the way the residuals method deals with the startup values (5 lags) versus the plot which extracts directly the standardized residuals used in the likelihood routine. This gives the same answer: par(mfrow=c(2,2)) plot(modelgarch,which=11) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi[-c(1:5)]) In most cases this is not likely be significantly different. It is related to a problem of how to initialize the ARMA recursion and what to return to the user. In THIS case, the first 5 values had an effect which changed the visual result marginally. For consistency I guess that the plot should return the result one would expect by using the residuals method on the model. Feel free to submit a patch. Regards, Alexios On 22/05/2013 13:40, Jen Bohold wrote:
It is not true, that I sent non-reproducible code/examples. I can give you the complete code again and I attach my data. You just have to run the code and you will see, that indeed both plots do differ. So this is replicatable. library(rugarch) myspecification<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) modelgarch<-ugarchfit(spec=myspecification,data=mydata) plot(modelgarch,which=11) library(forecast) resdi<-as.numeric(residuals(modelgarch,standardize=TRUE))^2 Acf(resdi) You can see, that the plots are NOT the same. I do not know, what is wrong with your code and I do not want to offend you, but there is clearly an error in it. ----- Urspr?ngliche Message ----- Von: alexios ghalanos <alexios at 4dscape.com> An: Jen Bohold <jenbohold at yahoo.de> CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org> Gesendet: 13:30 Mittwoch, 22.Mai 2013 Betreff: Re: Error in rugarch ACF squared standardized residuals plot I can't replicate your PNG chart differences and here is an example of how you can check: ########################################## library(rugarch) # you specification with YOUR estimated parameters: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0, ar4 = -0.292207, ar5 = -0.745887, ma1=0,ma2=0,ma3=0, ma4 = 0.309446, ma5 = 0.718856, omega = 6e-6, alpha1=0.093397, beta1 = 0.892404)) # Simulate a path sim=ugarchpath(modsp, n.sim=5000) # extract the simulated data mydata = as.numeric(fitted(sim)) # restate the specification: modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(5, 5), include.mean = FALSE), distribution.model = "norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0)) # estimate the model: modgarch<-ugarchfit(spec=modsp,data=mydata ) # extract the standardized residuals: resdi<-as.numeric(residuals(modgarch,standardize=TRUE)) # plot: library(forecast) par(mfrow=c(2,2)) Acf(resdi^2) plot(modgarch, which=11) ############################################### I don't see ANY differences. You are however welcome to look at the underlying code in the rugarch-plots.R file in the source. If you find a bug you are welcome to submit a patch to the google code repository of the package rather than continuously sending this list PNG attachments and non reproducible code/examples. Regards, Alexios On 22/05/2013 11:49, Jen Bohold wrote:
Dear Alexios,
thanks a lot for your response!
Yes, this was a typo, so I meant to write
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
and then plot it with
Acf(resdi^2)
this gives a DIFFERENT plot! It is NOT the same!
Again, I attach both plots.
----- Urspr?ngliche Message -----
Von: alexios ghalanos <alexios at 4dscape.com>
An: Jen Bohold <jenbohold at yahoo.de>
CC: "r-sig-finance at r-project.org" <r-sig-finance at r-project.org>
Gesendet: 11:43 Mittwoch, 22.Mai 2013
Betreff: Re: Error in rugarch ACF squared standardized residuals plot
Dear Jen,
The reason I have not answered is that you post one question, then
instead of patiently waiting for an answer, you very shortly post more
and more followups. As I said in a previous email, the likelihood of
answering, at least on my part, will depend on the effort shown to at
least try to do your own research and the framing of the question. You
also seem to be cross-posting to stackexchange.
With regards to your specific question, you are wrong and this is seen
by your own code:
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
This is NOT the standardized residuals of the model but the
observations, so that when you compare to the Acf plot you are comparing
the observations (before the estimation) to the standardized residuals
(after the ARMA filtration).
You probably wanted to write:
resdi<-as.numeric(residuals(modgarch,standardize=TRUE))
The plots of the results from rugarch are the same with what you get
with the Forecast package (which is actually a wrapper for the stats
package 'plot.acf').
I'm going to politely ask you to please take some more care when posting
and making such grand statement as "plot are not useable anymore". You
are quickly burning through any remaining goodwill left on the part of
this developer. Finally, I would suggest an excellent reference such as
Zivot and Wang ("Modeling Financial Time Series with S-PLUS") or Tsay
("Analysis of Financial Time Series") which may help you answer some of
your many questions.
Regards,
Alexios
On 22/05/2013 08:10, Jen Bohold wrote:
Although it seems that there is no feedback and you do not want to
comment on me, I thought I should share this to the list, maybe someone else
is some time wondering about this (maybe I did a mistake, but no one of the
list or you told me in the previous mail). Also, I do not want to offend
you, I like your package it's great! Especially I liked the acf plots, they
have a better design, although
you will see in the following text, that the "ACF of Squared
Standadrized Residuals" plot are not useable anymore.
The plot of the ACF of the squared standardized residuals in rugarch
output (you get it via plot(yourmodel) and choosing number 11) is wrong.
However, the corresponding Q-Statistics of the rugarch output are
correct!
Consider the following (I attached my data and the plots). I fitted the
following model (output extracted to the relevant parts):
*---------------------------------*
* GARCH Model Fit*
*---------------------------------*
Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model : ARFIMA(5,0,5)
Distribution : norm
Optimal Parameters
------------------------------------
Estimate Std. Error t value Pr(>|t|)
ar1 0.000000 NA NA NA
ar2 0.000000 NA NA NA
ar3 0.000000 NA NA NA
ar4 -0.292207 0.019550 -14.9467 0.0e+00
ar5 -0.745887 0.018488 -40.3436 0.0e+00
ma1 0.000000 NA NA NA
ma2 0.000000 NA NA NA
ma3 0.000000 NA NA NA
ma4 0.309446 0.026659 11.6073 0.0e+00
ma5 0.718856 0.021208 33.8952 0.0e+00
omega 0.000006 0.000001 4.2106 2.5e-05
alpha1 0.093397 0.011308 8.2591 0.0e+00
beta1 0.892404 0.012437 71.7563 0.0e+00
Q-Statistics on Standardized Residuals
------------------------------------
statistic p-value
Lag[1] 7.898 4.949e-03
Lag[p+q+1][11] 21.627 3.312e-06
Lag[p+q+5][15] 27.133 5.374e-05
d.o.f=10
H0 : No serial correlation
Q-Statistics on Standardized Squared Residuals
------------------------------------
statistic p-value
Lag[1] 1.274 0.258961
Lag[p+q+1][3] 9.351 0.002229
Lag[p+q+5][7] 12.135 0.032980
d.o.f=2
As you can see in the "Q-Statistics on Standardized Squared Residuals"
there is clearly correlation in the standardized squared residuals. BUT if
you look at the plot with the plot method and choosing number 11 you can
see, that NO spike is significant.
This plot is not correct, I controlled it via the Acf plot of the
forecast package and clearly, the spikes are larger! So the second spike is
now significant. I control the calculations via the Box.test method using
d.o.f.=2 and choosing the lag 3 and 7 and the calculations in the rugarch
package are correct! So the p-values are indeed 0.002229 and 0.032980. So
why is the plot of the rugarch package wrong?
One further notice: In a previous mail, I asked, why the lags in the
Q-Statistics on Standardized Squared Residuals are different to the lags
used in Q-Statistics on Standardized Residuals. Of course, I have now seen,
that the second uses the GARCH parameters, so it is clear, that this has to
be equal to two (1+1). I also have to say, that I think, that the ACF of
observations plot e.g. is indeed correct (number 4), so it seems, that the
plot number 11 uses different scaled residuals? Maybe it uses the
non-standardized squared residuals? Could that be the reason?
Thanks a lot for your notice.
My code:
library(rugarch)
modsp<-ugarchspec(variance.model = list(model = "sGARCH", garchOrder =
c(1, 1)),
mean.model = list(armaOrder = c(5, 5), include.mean = FALSE),
distribution.model =
"norm",fixed.pars=list(ar1=0,ar2=0,ar3=0,ma1=0,ma2=0,ma3=0))
modgarch<-ugarchfit(spec=modsp,data=mydata)
plot(modgarch)
residuals(mydata,standardize=TRUE)
resdi<-as.numeric(residuals(mydata,standardize=TRUE))
library(forecast)
Acf(resdi^2)
Box.test(resdi^2, lag = 3, type = "Ljung-Box", fitdf = 2)
Box.test(resdi^2, lag = 7, type = "Ljung-Box", fitdf = 2)
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
-- Peter Allington, M. Sc. CF&RM University of Washington
Peter Allington, M. Sc. CF&RM University of Washington
1 day later
On 22 May 2013 at 15:59, Peter Allington wrote:
| do not get confused: I answered with my friends account.
That is not tolerated here. We, just like the main R lists, strongly prefer that you
a) post under your real name and
b) preferably use a signature block providing additional identification.
Moreover, I happened to have been informed today that you have just been
thrown off two related, popular, non-email "Q and A" sites (I won't say more)
for exhibiting this type of behaviour (and then some).
We will not tolerate it either.
You have been in an excessive exchange with Alexios, who patiently took time
from actually working on his packages, to tell to a) slow down and not post
excessively and b) do some research yourself. You have not done either.
Please change your behaviour, or risk being ignored and/or expelled.
Dirk
(acting as listmaster)
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com