Hello, I have done some research about breakpoints (I am not a statistician) and I found out about the breakpoint, strucchange and segmented packages in R allowing to find breakpoints assuming linear model. However, I would like to fit a periodic time series with a non linear (periodic) model, and I was wondering how I could find breakpoints for this model in R. Is it even possible ? My model is an asymmetric gaussian fitting (cf http://www.nateko.lu.se/personal/Lars.Eklundh/Institutionssida/IEEE_TGRS_timesat.pdf) with a linear-time-dependant amplitude (I am fitting this model over the whole time series). *My ideas * 1) I guess I am more interested in the breakpoints of the "amplitude" of my periodic function, so that I could assume a model of the form: Y ~ (a + b*t)*f(t), with |f(t)| <= 1, where f is a periodic function to be fitted to a non linear model, but where no breakpoints should occur. So basically, the breakpoints would only happen in the (a,b) pair of coefficients, which would be a linear regression. However, as f is unknown, this makes things harder and I don't have a lot of extremas (min/max) to detect breakpoints robustly... 2) To detect breakpoint with an harmonic model and then to apply my non linear regression on each segment. These two ideas could potentially work, however these are workarounds. Thank you for your advices ! -- View this message in context: http://r.789695.n4.nabble.com/Breakpoints-and-non-linear-regression-tp4649072.html Sent from the R help mailing list archive at Nabble.com.
Breakpoints and non linear regression
4 messages · thomas88, Thomas Coquet, Achim Zeileis
On Fri, 9 Nov 2012, thomas88 wrote:
Hello, I have done some research about breakpoints (I am not a statistician) and I found out about the breakpoint, strucchange and segmented packages in R allowing to find breakpoints assuming linear model. However, I would like to fit a periodic time series with a non linear (periodic) model, and I was wondering how I could find breakpoints for this model in R. Is it even possible ? My model is an asymmetric gaussian fitting (cf http://www.nateko.lu.se/personal/Lars.Eklundh/Institutionssida/IEEE_TGRS_timesat.pdf) with a linear-time-dependant amplitude (I am fitting this model over the whole time series). *My ideas * 1) I guess I am more interested in the breakpoints of the "amplitude" of my periodic function, so that I could assume a model of the form: Y ~ (a + b*t)*f(t), with |f(t)| <= 1, where f is a periodic function to be fitted to a non linear model, but where no breakpoints should occur. So basically, the breakpoints would only happen in the (a,b) pair of coefficients, which would be a linear regression. However, as f is unknown, this makes things harder and I don't have a lot of extremas (min/max) to detect breakpoints robustly... 2) To detect breakpoint with an harmonic model and then to apply my non linear regression on each segment.
I would probably first try whether the following leads to reasonable fits Y(t) = A * exp(b * t) * H(t) i.e., a multiplicative model with an exponential trend and some harmonic trend. By taking logs you then get log Y(t) = log(A) + b * t + log(H(t)) -> log(Y(t)) = a + b * t + h(t) so that you can fit a model with a linear trend plus harmonic season to the log-series. And, of course, the harmonic trend can then be built up up sin/cos at different frequencies and you could fit the whole thing as a linear model to the log-series. It's not quite the same model that you propose above but might be an approach worth exploring. You could also look at the "bfast" package which has a function bfastpp() for setting up trend and harmonic season for a time series. And it also allows for iterative fitting of separate trend and season breakpoints in the time series. hth, Z
These two ideas could potentially work, however these are workarounds. Thank you for your advices ! -- View this message in context: http://r.789695.n4.nabble.com/Breakpoints-and-non-linear-regression-tp4649072.html Sent from the R help mailing list archive at Nabble.com.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20121109/49575683/attachment.pl>
On Fri, 9 Nov 2012, Thomas Coquet wrote:
Hello, I already tried and looked at the bfast package (very nice package by the way!) as I am working on VI time series as well.
Good! :-)
However, my model is definitely not linear,
Not even after taking logs or some other transformation? In principle, the breakpoint ideas can of course also be applied to non-linear models but so far in my applications I could always find transformations that lead rather naturally to roughly piecewise linear relationships.
so in worst case scenario my idea was to use the bfast package to find the breakpoints (with the harmonic fit) and then to fit the seasonal part in each segment with my model (so basically almost what you are suggesting - using harmonic to find breakpoints).
Yes, but for the log-transformed data...
But the breakpoints will not be dependent on my model, so this may be an issue, isn't it ?
Yes.
The asymmetric gaussian fit has been recognized as being one of the best fit for VI time series, and I used this method for periodic fit (so far it was used only as a smoothing function of the time series, not as a fit for the seasonal component).? The point would be to combine this method with an iterative breakpoint method such as bfast to detect abrupt changes, but to do that I need to find breakpoints in the seasonal trend with a non linear model (that is the tricky part :) ).
In principle, you can set up the same type of procedure that bfast uses with a non-linear model - as long as the objective function is additive in the observations. But I wouldn't know of a (fast enough) fitting function for such a segmented model in R. hth, Z
Thanks !
On Fri, Nov 9, 2012 at 2:00 PM, Achim Zeileis <Achim.Zeileis at uibk.ac.at>
wrote:
On Fri, 9 Nov 2012, thomas88 wrote:
Hello,
I have done some research about breakpoints (I am
not a statistician) and I
found out about the breakpoint, strucchange and
segmented packages in R
allowing to find breakpoints assuming linear model.
However, I would like to fit a periodic time series
with a non linear
(periodic) model, and I was wondering how I could
find breakpoints for this
model in R. Is it even possible ?
My model is an asymmetric gaussian fitting (cf
http://www.nateko.lu.se/personal/Lars.Eklundh/Institutionssida/IEEE_TGRS_ti
mesat.pdf)
with a linear-time-dependant amplitude (I am fitting
this model over the
whole time series).
*My ideas
*
1) I guess I am more interested in the breakpoints
of the "amplitude" of my
periodic function, so that I could assume a model of
the form:
Y ~ (a + b*t)*f(t), with |f(t)| <= 1, where f is a
periodic function to be
fitted to a non linear model, but where no
breakpoints should occur.
So basically, the breakpoints would only happen in
the (a,b) pair of
coefficients, which would be a linear regression.
However, as f is unknown,
this makes things harder and I don't have a lot of
extremas (min/max) to
detect breakpoints robustly...
2) To detect breakpoint with an harmonic model and
then to apply my non
linear regression on each segment.
I would probably first try whether the following leads to
reasonable fits
Y(t) = A * exp(b * t) * H(t)
i.e., a multiplicative model with an exponential trend and some
harmonic trend. By taking logs you then get
log Y(t) = log(A) + b * t + log(H(t))
->
log(Y(t)) = a + b * t + h(t)
so that you can fit a model with a linear trend plus harmonic
season to the log-series. And, of course, the harmonic trend can
then be built up up sin/cos at different frequencies and you
could fit the whole thing as a linear model to the log-series.
It's not quite the same model that you propose above but might
be an approach worth exploring. You could also look at the
"bfast" package which has a function bfastpp() for setting up
trend and harmonic season for a time series. And it also allows
for iterative fitting of separate trend and season breakpoints
in the time series.
hth,
Z
These two ideas could potentially work, however
these are workarounds.
Thank you for your advices !
--
View this message in context:http://r.789695.n4.nabble.com/Breakpoints-and-non-linear-regression-tp46490
72.html
Sent from the R help mailing list archive at
Nabble.com.
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained,
reproducible code.