Sorry Brian, I misunderstood your last reply.
Intent: Evaluate forecasting accuracy of autoregressive models
Approach: Apply Superior Predictive Ability (SPA) test in the ?ttrTests?
package to assess the forecasting accuracy
Version: R ver. 3.0.2 (2013-09-25) - "Frisbee Sailing"
Editor: R Studio ver. 0.98.1091
rugarch package: 1.3-4
ttrTests package: 1.7
Code: http://dropbox.unl.edu/uploads/20141219/e61f31203e15e90d/sample.zip
I am lost and looking for some direction on how to use a loss function
series to evaluate the efficacy of volatility forecasts between models
fm1 through fm5 using the SPA test in the "ttrTests" package.
My attempt is as follows:
spa <- dataSnoop(MeanSquareError,bSamples=1000,test="SPA")
# The structure of "MeanSquareError" loss function series is as follows:
Day msefm1 msefm2 .. msefm5
T+1 0.0000464 0.0000411 .. 0.0000407
T+2 0.0000461 0.0000421 .. 0.0000428
...
T+49 0.0000174 0.0000653 .. 0.0001126
T+50 0.0000174 0.0000657 .. 0.0001138
My code is structured as follows:
# To load the sample data in the "rugarch" package
data(sp500ret)
# To specify the desired autoregressive models
model1 = ugarchspec(variance.model = list(model = "sGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model2 = ugarchspec(variance.model = list(model = "gjrGARCH",garchOrder
= c(1, 1)), distribution.model = "std")
model3 = ugarchspec(variance.model = list(model = "sGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model4 = ugarchspec(variance.model = list(model = "csGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model5 = ugarchspec(variance.model = list(model = "apARCH",garchOrder =
c(1, 1)), distribution.model = "std")
# To estimate the autoregressive models specified
m1 = ugarchfit(sp500ret, spec = model1)
m2 = ugarchfit(sp500ret, spec = model2)
m3 = ugarchfit(sp500ret, spec = model3)
m4 = ugarchfit(sp500ret, spec = model4)
m5 = ugarchfit(sp500ret, spec = model5)
# To forecast the volatility for next 50 days with the parameters
estimated above
fm1 = ugarchforecast(m1,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm2 = ugarchforecast(m2,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm3 = ugarchforecast(m3,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm4 = ugarchforecast(m4,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm5 = ugarchforecast(m5,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
I used the extractor function "sigma" in the "rugarch" package to
extract the conditional standard deviation of the forecasts fm1 through
fm5, which is then fed to a spreadsheet to estimate the Mean Square
Error (MSE) loss function series for the 50 days.
# The MSE loss function series is then imported into R.
MeanSquareError <- read.csv("F:/MeanSquareError.csv")
Now, how do I go about using this ?MeanSquareError? loss function series
such as this: spa<-dataSnoop(MeanSquareError, bSamples=1000,
test="SPA")? Please note that I am not looking to convert
?MeanSquareError.csv? into R-data, rather I am searching for a way to
apply the SPA test.
Thank you for your time.
Karthik
Brian G. Peterson wrote on 2014-11-29 03:51 PM +0530:
I asked you a reproducible example, and instead, you're asking the list
to do a bunch of work to replicate your code for you.
Please respect the time of the list members, and provide a few lines of
code, so that we can *run* your example code and then *help you* with
your problem.
Brian
On 11/28/2014 09:44 PM, Karthik Raju wrote:
Hello Brian,
Thank you for your response.
I will explain my work with an example.
- I have logarithmic returns computed from daily closing values of a
stock market index for a period of 500 days.
- The logarithmic returns are used in the "rugarch" package to fit the
following models,
Model-1 - GARCH (1,1)
Model-2 - GJR-GARCH (1,1)
Model-3 - EGARCH (1,1)
Model-4 - IGARCH (1,1)
Model-5 - APARCH (1,1)
- Then, using "rugarch", forecasted one step ahead volatility with each
of the above model parameters for an out-of-sample period of 100 days.
- The forecasted 100 days "sigma" values of each model is extracted from
R to a worksheet and used against true volatility proxy to compute the
loss functions.
- The structure of loss function data for each model is an univariate
series for the forecasted period (100 days).
- For example, the structure of Mean Square Error (MSE) loss function
looks like,
Forecasts MSE_GARCH MSE_GJR . . MSE_APARCH
Day 1 0.008548 0.005509 . . 0.00412
Day 2 0.008655 0.005498 . . 0.00414
Day 3 0.008759 0.005488 . . 0.004156
Day 4 0.008861 0.005478 . . 0.004169
. . . . . .
. . . . . .
. . . . . .
Day 99 0.008759 0.005488 . . 0.004156
Day 100 0.008861 0.005478 . . 0.004169
- Similarly, the loss functions like Mean Absolute Error, Mean Absolute
Percentage Error etc. are in the same structure, which are then imported
into R.
- Now, by using each of this loss function data series, one at a time,
in the "ttrTests" package, I want to know how to compute the SPA test
for the following combinations?
Benchmark model GARCH (1,1) Vs rest of 4 models
Benchmark model GJR GARCH (1,1) Vs rest of 4 models
Benchmark model EGARCH (1,1) Vs rest of 4 models
Benchmark model IGARCH (1,1) Vs rest of 4 models
Benchmark model APARCH (1,1) Vs rest of 4 models
Best,
Karthik
Please follow the posting guide
http://www.r-project.org/posting-guide.html
and provide a reproducible example. For instance, you could repeat your
'n' samples using some of the test data from rugarch.
It is very difficult for others to help you if we don't know the format
your data is in *now*.
Regards,
Brian
On 11/27/2014 11:24 PM, Karthik Raju wrote:
Hello,
Can anyone please let me know how to prepare data to compute Superior
predictive ability (SPA) test in R?
I am working on forecasting volatility in stock markets, the context is,
1) I used "rugarch" package to forecast the volatility using "n"
autoregressive models.
3) Extracted "sigma" values from the forecasts of those "n" models and
used it in a worksheet against realized volatility to calculate daily
loss functions for each model, say, MSE, MAE, RMSE etc.
3) I want to compute the SPA test statistics using "ttrTests" package
and find the SPA test p-values for a benchmark model against "n"
alternatives. How to prepare data for this purpose?
Thanks,
Karthik
_______________________________________________
R-SIG-Finance at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.