LPPL model for bubble burst forcasting

Hello Wind,
I was wondering if you made any progress on the R implementation of the
model, and if you could/would be willing to share what you got so far?

Regards,

-Mark-

2009/7/24 Wind <windspeedo99 at gmail.com>

Since I am not good at coding or statistics, genetic algo has been used
instead of tabu search.    Package rgenoud is great and idiot proof.

Attached is the latest analysis on SSEC index, the equity market index of
mainland China.  genoud() has been run for 10 times, generating 10 fits.
The residuals are all stationary according to ADF test from package urca.
But the time window setting is subjective and maybe some minor problems on
parameter conditions such as omega and C.  So,  just for fun.

Thanks for all the encouragement and help from the list.    And thanks for
the detailed instructions on statistics issues by one of the author on LPPL,
Dr. Lin.

wind

On Mon, Jul 20, 2009 at 9:22 PM, Brian G. Peterson <brian at braverock.com>wrote:

Glad to see you're making progress on this problem.

This paper
http://www.diegm.uniud.it/satt/papers/DiSc06b.pdf

implements a tabu search in R, though they didn't publish their code.
 You might want to contact them for the implementation and permission to
share their tabu search algorithm/code with the R community.

They also reference an R package called RACE by this gentleman:
http://iridia.ulb.ac.be/~mbiro/
that they use for evaluating the solution.

More generally, from my limited understanding, tabu search is an
extension and refinement of simulated annealing approaches easily
implemented in R.  In brief the approach is to take your best 'n' solutions
from a random space search, and then search "near" those solutions.
 Simulated annealing and its close cousins have a lot of benefits in
finance, where a single true optima from a closed form problem is not likely
to be available.  I personally have rarely found 'optim' to be usable for my
problem space, and have had to use other solvers for practical problems in
finance.

Regards,

 - Brian


Wind wrote:

Some progress.   The LPPL curve could be plotted with the following
codes.
The problem now is how to get the best fit parameters.
Some researchers  use python or matlab for LPPL calibrating.     It
seems that some of them prefer tabu search for optimums locating.   It
seems that there's still no general function for tabu search in R.
At the end of codes, I give the possible parameter combinations to be
searched in, maybe there are other functions for optimum searching in
R.
Any suggestion would be appreciated.


## Financial Bubbles, Real Estate bubbles, Derivative Bubbles, and the
Financial and Economic Crisis
## http://arxiv.org/abs/0905.0220
## Fig. 23 S&P500 index (in logarithmic scale)in  Page 39


library(quantmod)

LPPL1<-function(p,dtc=20,alpha=0.35,omega=0.1,phi=1)
{
       #function in page 26 of http://arxiv.org/abs/0905.0220
       #the basic form of LPPL
       dtc=abs(floor(dtc))
       tc<-length(p)+dtc
       dt<-abs(tc-(1:length(p)))

       x1<-dt^alpha
       x2<-(dt^alpha)*cos(omega*log(dt)+phi)

       f<-lm(log(p) ~ x1+x2)


 f$para<-list(recno=dim(f$model)[1],dtc=dtc,alpha=alpha,omega=omega,phi=phi,sigma=summary(f)$sigma)
       return(f)
}

opt.lppl<-function(x)
{
       #derived function for optim

 return(LPPL1(p,dtc=x[1],alpha=x[2],omega=x[3],phi=x[4])$para$sigma)
}

LPPL1.x<-function(lpplf,pt=100)
{
       #x axis for predicting
       dt<-abs((lpplf$recno+lpplf$dtc)-(1:(lpplf$recno+lpplf$dtc+pt)))
       dt[dt==0]<-0.5

       x1<-dt^lpplf$alpha
       x2<-(dt^lpplf$alpha)*cos(lpplf$omega*log(dt)+lpplf$phi)
       return(list(x1=x1,x2=x2))

}

#get the SP500 index

pr<-getSymbols("^GSPC",auto.assign=FALSE,from="2003-10-1",to="2007-05-16")[,4]
p<-as.numeric(pr)


plot(p,type="l",log="y",xlim=c(0,length(p)+100),ylim=c(min(p),max(p)*1.2))
abline(v=length(p),col="green")

#something like the Fig. 23 in  Page 39 of
http://arxiv.org/abs/0905.0220
#but obviously the result is not calibrated well
#using optim like this can not calibrate the LPPL model
opts<-sapply(seq(1,50,10),function(x){
                       o<-optim(c(x,0.6,20,1),opt.lppl)

 f3<-LPPL1(p,dtc=o$par[1],alpha=o$par[2],omega=o$par[3],phi=o$par[4])
                       xp<-LPPL1.x(f3$para,200)
                       f3p<-predict(f3,data.frame(x1=xp$x1,x2=xp$x2))
                       lines(exp(f3p),col="blue")
                       lines(exp(fitted(f3)),col="red")
                       f3$para
               })



##crash point after dtc days
dtc<-seq(1,100,1)

##appropraite range of the parametes of LPPL
##according to Dr. W.X. Zhou's new book which is in Chinese
##the increments of the sequences are added according to my own judement
alpha<-seq(0.01,1.2,0.1)
omega<-seq(0,40,1)
phi<-seq(0,7,0.1)

##millions possible combinations
#complete test would be difficult
para<-expand.grid(dtc=dtc,alpha=alpha,omega=omega,phi=phi)
dim(para)

system.time(sigs<-apply(para[1:100,],1,function(x){LPPL1(p,dtc=x[1],alpha=x[2],omega=x[3],phi=x[4])$para$sigma}))

##methods for minimum sigma searching within the parameter combinations
##not implemented yet


wind





On Thu, Jul 16, 2009 at 9:25 PM, Brian G. Peterson<brian at braverock.com>
wrote:

So first, using your real name and ideally your professional identity,
ask
for the python code.  Better yet, get an academic buddy to do it.
Usually
getting access to the code isn't too tough.  Mention things like
"repeatable
research" and "collaboration" in your email.  Two of the authors
publish
their email addresses in one of the papers you reference, so contacting
them
should be easy.

Next port the python code to R.

If you can't do that, then replicate the model in R "from scratch".  A
trivial scan of the paper in question lends several techniques that are
well
covered in R: AR, GARCH, power laws, linear regression, stochastic
discount
factor, Ornstein-Uhlenbeck, etc.
There are volumes of information available on these topics from within
R, in
numerous books, and in the archives of this mailing list and r-help.

You're going to have to do your replication in pieces, probably
starting
with their implementation of the log periodic power law (LPPL), for
which I
do not believe there is an existing direct analogue in R though all the
component parts necessary to replicate it should be readily available.

As you work on each step of the replication, share your code with this
list
and the problems you are having with a particular step.  Ask specific,
directed questions with code to back them up.  Someone will likely help
you
solve the specific problem.

In R generally, it is not necessary that you be able to *do* the math
(think
pencil and paper), but if you plan to replicate published work, it will
be
necessary to *understand* at least some of how the math works, and to
be
able to pick out the names of techniques that you can search for an
utilize.

Basically, I'm recommending that you (specifically) and others (more
generally) should share the process of replicating a technique like
this, as
well as the final product, to give all the rest of us who are likely to
be
helping "you" get all this done. quid pro quo.

Cheers,

 - Brian


--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock



Wind wrote:

Prof. Sornette has spent years forcasting bubble burst with
"log-periodic power law".    The latest paper  gives "a
self-consistent model for explosive financial bubbles, which combines
a mean-reverting volatility process and a stochastic conditional
return which reflects nonlinear positive feedbacks and continuous
updates of the investors' beliefs and sentiments."

And his  latest  predicting is the burst of Chinese equity bubble at
the end of July.     http://arxiv.org/abs/0907.1827

While waiting to see the result, I wonder whether it is possible to
replicate the forcast with R.  The model is in the page 10 of the "A
Consistent Model of `Explosive' Financial Bubbles With Mean-Reversing
Residuals",  http://arxiv.org/abs/0905.0128  .   The output chart is
in the page 3 of "The Chinese Equity Bubble: Ready to Burst",
http://arxiv.org/abs/0907.1827 .   I guess the authors of the latter
paper use the same model as described in the first paper.

Because statistics is still challenging for me though I could use R
for  basic data manipulations,  I wonder which package or function
would be necessary to implement the model in the paper.  The model
seems more complicated than the models in the R tutorials for me.
By the way, the author of the paper used Python and the codes are
private.

Any suggestion would be highly appreciated.

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock