problem aggregating daily series
Dear John Frain,
please apologize for any inconveniances caused by a bug in the
applySeries() Function.
It doesent appear when you worj with daily records using "GMT" as FinCenter.
Try:
myFinCenter = "GMT"
tD = timeSequence("2006-01-01", length.out = 365)
Data = matrix(rep(1, times = 365), ncol = 1)
tS = timeSeries(Data, tD)
from = timeCalendar(2006)
to = timeLastDayInMonth(from)
applySeries(tS, from, to, FUN = colSums, units = "N")
Now fix the bug by overwriting the last line in applySeries()
timeSeries(data = rowBind, charvec = as.character(to), units = units,
# format = x at format, zone = "GMT", FinCenter = x at FinCenter,
format = x at format, zone = x at FinCenter, FinCenter = x at FinCenter,
title = x at title, documentation = x at documentation, ...)
Here is the full function:
.applySeries =
function (x, from = NULL, to = NULL, FUN = colAvgs, units = NULL, ...)
{
colNames = units
if (class(x) != "timeSeries") stop("s is not a timeSeries object")
fun = match.fun(FUN)
j.pos = as.POSIXct(seriesPositions(x))
j.from = as.POSIXct(from)
j.to = as.POSIXct(to)
y = x at Data
pos = seriesPositions(x)
rowNames = rownames(x at Data)
rowBind = NULL
for (i in 1:from at Dim) {
test = (j.pos >= j.from[i] & j.pos <= j.to[i])
cutted = as.matrix(y[test, ])
ans = fun(cutted, ...)
rowBind = rbind(rowBind, ans)
}
rownames(rowBind) = as.character(to)
if (is.null(colNames)) {
units = x at units
}
else {
units = colNames
}
timeSeries(data = rowBind, charvec = as.character(to), units = units,
# format = x at format, zone = "GMT", FinCenter = x at FinCenter,
format = x at format, zone = x at FinCenter, FinCenter = x at FinCenter,
title = x at title, documentation = x at documentation, ...)
}
# Now try:
myFinCenter = "Dublin"
tD = timeSequence("2006-01-01", length.out = 365, FinCenter = "Dublin")
Data = matrix(rep(1, times = 365), ncol = 1)
tS = timeSeries(Data, tD, zone = "Dublin", FinCenter = "Dublin")
from = timeCalendar(2006)
to = timeLastDayInMonth(from, FinCenter = "Dublin")
.applySeries(tS, from, to, FUN = colSums, units = "N")
The philosophy behind the timeSeries class is that you have two
time reference points: The first where the data were collected (zone)
and the
second where the data willl be used (FinCenter). To circumvent
inconsistencies
in the future, I think it will be the best to allow to specify in any
case zone and
FinCenter through the argument list in any of the timeSeries class
functions
Diethelm Wuertz
John C. Frain wrote:
I am having problems converting daily returns to monthly. The output of one of the many various attempts that I have made is given below. daily.csv is a csv file with dates -in the format "$Y-$m-%d" with daily returns on 29 equities in the columns. I do not appear to be able to generate correct results. In the final example below the aggregated data for October to February are correct. The aggregated data for March to September exclude the last day of the month and give the wrong answer (unless the return on that last day is zero or the last day is at the weekend). It is likely that the problem is caused by my not accounting for summer time correctly. I am probably missing something obvious and would be delighted if someone could enlighten me. <to = timeLastDayInMonth(as.character.timeDate(from))> as used below is one of several formulations that give an error message with the applySeries. In another formulations of the to dates adding or subtracting a second removed the error. My work around for this problem is to code the aggregation directly using the applySeries function as a model. Any suggestions would be welcome. Ialso note that in example fCalender\demo\xmpZWChapter02.R in the Rmetrics distribution Diethelm Wuertz mentions two arguements "include.from" and "include.to" which determines if the start or end/date/time are included or not with a default of including the start and excluding the end. (Rmetrics equivalent of material p. 34 of Zivot and Wang). Including a logical "include.to=TRUE" causes an error. I have updated my fBasics and fCalendar libraries recently. Again any comments would be welcome The output (with some material removed) of my most recent program is below. Conversions were checked directly in R using instructions such as sum(cutSeries(T.ts3,"1990-01-01","1990-01-31")@Data) for a variety of series and then verified in RATS. The output (with some material removed) of my most recent program is below. R : Copyright 2006, The R Foundation for Statistical Computing Version 2.3.1 (2006-06-01) ISBN 3-900051-07-0 Loading required package: datasets Loading required package: utils Loading required package: grDevices Loading required package: graphics Loading required package: stats Loading required package: methods Loading required package: tcltk Loading Tcl/Tk interface ... done Loading required package: R2HTML Loading required package: svMisc Loading required package: svIO Loading required package: svViews During startup - Warning message: use of NULL environment is deprecated
library(fBasics)
Rmetrics, (C) 1999-2005, Diethelm Wuertz, GPL fBasics: Markets and Basic Statistics
library(fCalendar)
Rmetrics, (C) 1999-2005, Diethelm Wuertz, GPL fCalendar: Time, Date and Calendar Tools
myFinCenter="Europe/Dublin" # # read in data from csv file to timeseries object daily.ts=read.timeSeries(file="D:\\equities\\yahoo\\R\\daily_data\\daily.csv",
+ zone="GMT",FinCenter=myFinCenter,sep=",")
#head(daily.ts) #tail(daily.ts) # #set up from and to series for aggregation y = c(rep(1990,12),rep(1991,12),rep(1992,12),rep(1993,12),rep(1994,12),
+ rep(1995,12),rep(1996,12),rep(1997,12),rep(1998,12),rep(1999,12),rep(2000, 12), + rep(2001, 12),rep(2002, 12),rep(2003, 12),rep(2004, 12),rep(2005, 12), + rep(2006, 9))
m = c(rep(1:12,16),1:9) from = timeCalendar(y, m,FinCenter=myFinCenter)[1:200] to = timeLastDayInMonth(as.character.timeDate(from)) data.frame(from, to)[1:5,]
from to 1 1990-01-01 00:00:00 1990-01-31 00:00:00 2 1990-02-01 00:00:00 1990-02-28 00:00:00 3 1990-03-01 00:00:00 1990-03-31 00:00:00 4 1990-04-01 00:00:00 1990-04-30 01:00:00 5 1990-05-01 01:00:00 1990-05-31 01:00:00
ndays=from at Dim data.frame(from,to)[(ndays-5):ndays,]
from to 195 2006-03-01 00:00:00 2006-03-31 00:00:00 196 2006-04-01 00:00:00 2006-04-30 01:00:00 197 2006-05-01 01:00:00 2006-05-31 01:00:00 198 2006-06-01 01:00:00 2006-06-30 01:00:00 199 2006-07-01 01:00:00 2006-07-31 01:00:00 200 2006-08-01 01:00:00 2006-08-31 01:00:00
# Aggregate to Monthly Means: monthly.ts=applySeries(daily.ts, from = from, to = to, FUN = colSums)
Error in if (timeTest == 0) iso.format = "%Y-%m-%d" :
missing value where TRUE/FALSE needed
#
write.csv(file="D:\\equities\\yahoo\\R\\daily_data\\monthly_normality.csv",monthly.ts at Data) Error in inherits(x, "data.frame") : object "monthly.ts" not found
#
# Direct implementation apply Series
#
#
j.pos = as.POSIXct(seriesPositions(daily.ts))
j.from =as.POSIXct(from)
j.to = as.POSIXct(to)
y = daily.ts at Data
pos = seriesPositions(daily.ts)
rowNames = rownames(daily.ts at Data)
rowBind = NULL
for (i in 1:from at Dim) {
+ test = (j.pos >= j.from[i] & j.pos <= j.to[i]) + cutted = as.matrix(y[test, ]) + ans = colSums(cutted) + # print(dim(ans)) + rowBind = rbind(rowBind, ans) + # print(dim(rowBind)) + }
rownames(rowBind) = as.character(to)
units = daily.ts at units
monthly.ts=timeSeries(data = rowBind, charvec = as.character(to), units =
units, + zone = "GMT", FinCenter = daily.ts at FinCenter, + title = daily.ts at title, documentation = daily.ts at documentation)
write.csv(file="D:\\equities\\yahoo\\R\\agreg\\monthly.csv",monthly.ts at Data)
John C. Frain. Economics Department Trinity College Dublin Dublin 2 www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj at tcd.ie
_______________________________________________ R-SIG-Finance at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance