In highfrequency package `convert` function creates folder structure, but no .RData file with TickData.com data
Nick,
On Thu, Sep 18, 2014 at 12:41 AM, Nick White <n-e-w at qtradr.net> wrote:
Hi all, I also raised the following (summarised, but pretty much verbatim) as an issue on the highfrequency package's github Issues page... but that doesn't appear to get a lot of attention: https://github.com/jonathancornelissen/highfrequency/issues/3
I think the github account was only used for the initial Google Summer of Code project. The current code base seems to be on R-Forge: https://r-forge.r-project.org/projects/highfrequency/
Here's the same substance: I am new to using the package. The following issue is with TickData.com files. I have setup the folders with a single day's worth of data for XYZ as per the documentation: from = "2014-09-02"; to = "2014-09-02"; datasource = "~/raw_data"; datadestination = "~/xts_data" in ~/raw_data I have placed the files: XYZ_quotes.asc XYZ_trades.asc which are simply copied and renamed files from my original TickData folder. Note, I have *not* extracted these using TickWrite7. These are just the raw .asc files from the download directory. I then run the convert function as follows:
convert(from=from, to=to, datasource=datasource, datadestination=datadestination, trades = TRUE,quotes = TRUE, ticker="XYZ", dir = TRUE, format = "%d/%m/%Y %H:%M:%OS",extension = "tickdatacom", header = TRUE, onefile = TRUE )
which begins to run and take some time....but then produces this error: Error in if (length(c(year, month, day, hour, min, sec)) == 6 && c(year, : missing value where TRUE/FALSE needed In addition: Warning messages: 1: In as_numeric(YYYY) : NAs introduced by coercion 2: In as_numeric(YYYY) : NAs introduced by coercion which, when calling traceback produces:
traceback()
7: (function (year = 1970, month = 12, day = 31, hour = 23, min = 59,
sec = 59, subsec = 0.99999, tz = "")
{
if (!missing(sec) && sec%%1 != 0)
subsec <- 0
sec <- ifelse(year < 1970, sec, sec + subsec)
mon.lengths <- c(31, 28, 31, 30, 31, 30, 31, 31, 30, 31,
30, 31)
if (missing(day)) {
day <- ifelse(month %in% 2, ifelse(((year%%4 %in% 0 &
!year%%100 %in% 0) | (year%%400 %in% 0)), 29, 28),
mon.lengths[month])
}
if (length(c(year, month, day, hour, min, sec)) == 6 && c(year,
month, day, hour, min, sec) == c(1969, 12, 31, 23, 59,
59) && Sys.getenv("TZ") %in% c("", "GMT", "UTC"))
sec <- sec - 1
ISOdatetime(year, month, day, hour, min, sec, tz)
})(year = NA_real_, tz = "GMT")
6: do.call(lastof, parse.side(intervals[2], intervals[1]))
5: as.POSIXlt(do.call(lastof, parse.side(intervals[2], intervals[1])))
4: .parseISO8601(ii, .index(x)[1], .index(x)[nr], tz = tz)
3: `[.xts`(alldata, as.character(dates[i]))
2: alldata[as.character(dates[i])]
1: convert(from = from, to = to, datasource = datasource,
datadestination = datadestination,
trades = TRUE, quotes = TRUE, ticker = "XYZ", dir = TRUE,
format = "%d/%m/%Y %H:%M:%OS", extension = "tickdatacom",
header = TRUE, onefile = TRUE)
It's really hard to say without a reproducible example, but the error occurs either on line 2301 or 2308 of realized.R. And dates is defined as: dates = unique(as.Date(index(alldata))) which makes me think this is a TZ issue. The default TZ for as.Date is "UTC", but the TZ of alldata is likely different. You could cross your fingers and run Sys.setenv(TZ="UTC") at the beginning of your R session and see if that works...
This occurs using extracted .asc files from TickWrite7 both where Date and Time field is chosen or where individual Date and Time fields were selected. This also happens whether I extract the files using TW7 or just get use the raw files in my TickWrite data download folders as noted above. This is on R3.0.2, x64, Win8.1P, highfrequency v = 0.3 Any help v much appreciated. Cheers, n.
Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com