Skip to content

exponentially weighted linear regression

11 messages · Zachary Mayer, riccardo visca, Eric Zivot +1 more

Original
4 days later
Eric Zivot
# 
It seems like this can be easily solved using rollApply or period.apply and
lm with exponentially declining weights (in the w option). BTW, regression
on exponentially weighted data is often called "discounted least squares"
and is what I would call a "poor man's" kalman filter. In fact there have
been a few papers in the engineering literature which shows that discounted
least squares is equivalent to a certain type of filtered estimated from a
state space regression model with time varying parameters. I've used
discounted least squares for modeling hedge fund data and found it to work
quite nicely. Also, the time varying parameter state-space model may work
well with a few explanatory variables (1 or 2) but it often does not perform
well if there are many (say 5+ predictors). For each regression parameter,
you have to estimate an AR(1) smoothing parameter plus a transition equation
error term. With many predictors, you run into numerical stability problems
very quickly and the likelihood function can have many local minima. For
this reason, discounted least squares is an attractive alternative. Finally,
it is very difficult to do model selection with the state space kalman
filter. How do you choose the variables to enter the regression equation? I
have not seen anyone do a systematic study of model selection in time
varying parameter models. Perhaps, this is where a Bayesian approach might
be useful. This is an important but neglected topic. I would be happy is
someone pointed me to some research on this topic.


Eric Zivot                  			               
Robert Richards Chaired Professor of Economics and Director of Outreach
Adjunct Professor of Finance                            
Adjunct Professor of Statistics
Adjunct Professor of Applied Mathematics 
Department of Economics
Box 353330                  email:  ezivot at u.washington.edu 
University of Washington    phone:  206-543-6715            
Seattle, WA 98195-3330
www:  http://faculty.washington.edu/ezivot                  




-----Original Message-----
From: r-sig-finance-bounces at r-project.org
[mailto:r-sig-finance-bounces at r-project.org] On Behalf Of riccardo visca
Sent: Thursday, January 05, 2012 7:44 AM
To: Zachary Mayer; Michael
Cc: r-sig-finance
Subject: Re: [R-SIG-Finance] exponentially weighted linear regression

I think you should use a time varying parameter model (look at dlm package)
or if you do not have time to learn try to smooth the predictors first then
run the rolling window regression maybe robustified.

If you use the Kalman filter use an AR(1) for the parameter in the state
equation like

yt = bt * Xt-1
bt = c * bt-1

c will do the smoothing (it is your (1-lambda))

        

      
      
    

  
  


  


      

    

    
1 day later