data handling with timeSeries - R-SIG-Finance

Wed, Dec 3, 2008 6:45 AM #

Dear All,

I met some problems with data handling when using the timeSeries library

I have some csv files, each containing two columns, first column are the series of dates, second are the series prices 

I read in all files using, and create a matrix with the dates in the first column, and the calculated returns in the remaining columns

And then I would like to have the last 1000 lines of this matrix for some computations

But

shows that past1000 doesn't contain either the column of dates, nor the header line, only the columns of returns  

Also

returns FALSE, and although I attempted several conversions on past1000, which resulted this check to be TRUE, the effective content of past1000 didn't seem to be changed, still no dates were printed

And despite

returns TRUE, and printing the whole ralls object contains the dates column and the header

returns only the second column of ralls (that is, the first column of returns) and no date column is displayed. This seems to indicate that perhaps there is already some problem with ralls, rooted possibly in the datafile reading and binding?           

The expectation would be to have past1000 an object where the first column contains the dates, and the remaining columns contain the returns, but I couldn't figure it out so far.

Any suggestion is very much appreciated.

Best regards,
/Johan

Jeff Ryan

Wed, Dec 3, 2008 7:21 AM #

Try:

past1000 <- ralls[(index-1000):(index-1),   ]   # the comma is critical

GMT
            x
2008-12-04  1
2008-12-05  2
2008-12-06  3
2008-12-07  4
2008-12-08  5
2008-12-09  6
2008-12-10  7
2008-12-11  8
2008-12-12  9
2008-12-13 10

[1] 1 2 3

GMT
           x
2008-12-04 1
2008-12-05 2
2008-12-06 3

HTH
Jeff

On Wed, Dec 3, 2008 at 8:45 AM, J Konnonen <jkonn2001 at yahoo.com> wrote:

Dear All,

I met some problems with data handling when using the timeSeries library

I have some csv files, each containing two columns, first column are the series of dates, second are the series prices

I read in all files using, and create a matrix with the dates in the first column, and the calculated returns in the remaining columns

library(timeSeries)

returnsdata <- read.csv("foo1.csv",header=T,sep=",")
attach(returnsdata)
foo1 = as.timeSeries(returnsdata)
foo1bind <- returns(window(foo1, start=2007-01-01, end=2007-12-31))

returnsdata <- read.csv("foo2.csv",header=T,sep=",")
attach(returnsdata)
foo2 = as.timeSeries(returnsdata)
foo2bind <- returns(window(foo2, start=2007-01-01, end=2007-12-31))

ralls <- cbind(foo1bind,foo2bind)
colnames(ralls) <- c("FOO1","FOO2")

And then I would like to have the last 1000 lines of this matrix for some computations

index <- ... #some value
past1000 <- ralls[(index-1000):(index-1)]

But

print(past1000)

shows that past1000 doesn't contain either the column of dates, nor the header line, only the columns of returns

Also

print(is.timeSeries(past1000))

returns FALSE, and although I attempted several conversions on past1000, which resulted this check to be TRUE, the effective content of past1000 didn't seem to be changed, still no dates were printed

And despite

print(is.timeSeries(ralls))

returns TRUE, and printing the whole ralls object contains the dates column and the header

print(ralls at .Data[,1])

returns only the second column of ralls (that is, the first column of returns) and no date column is displayed. This seems to indicate that perhaps there is already some problem with ralls, rooted possibly in the datafile reading and binding?

The expectation would be to have past1000 an object where the first column contains the dates, and the remaining columns contain the returns, but I couldn't figure it out so far.

Any suggestion is very much appreciated.

Best regards,
/Johan

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Jeffrey Ryan
jeffrey.ryan at insightalgo.com

ia: insight algorithmics
www.insightalgo.com

Gabor Grothendieck

Wed, Dec 3, 2008 7:33 AM #

Also tail(z, 1000) gives the last 1000 points of zoo object z.

On Wed, Dec 3, 2008 at 10:21 AM, Jeff Ryan <jeff.a.ryan at gmail.com> wrote:

Try:

past1000 <- ralls[(index-1000):(index-1),   ]   # the comma is critical

z <- zoo(1:10, Sys.Date()+1:10)
as.timeSeries(z)

GMT
           x
2008-12-04  1
2008-12-05  2
2008-12-06  3
2008-12-07  4
2008-12-08  5
2008-12-09  6
2008-12-10  7
2008-12-11  8
2008-12-12  9
2008-12-13 10

as.timeSeries(z)[1:3]

[1] 1 2 3

as.timeSeries(z)[1:3,]

GMT
          x
2008-12-04 1
2008-12-05 2
2008-12-06 3

HTH
Jeff


On Wed, Dec 3, 2008 at 8:45 AM, J Konnonen <jkonn2001 at yahoo.com> wrote:

Dear All,

I met some problems with data handling when using the timeSeries library

I have some csv files, each containing two columns, first column are the series of dates, second are the series prices

I read in all files using, and create a matrix with the dates in the first column, and the calculated returns in the remaining columns

library(timeSeries)

returnsdata <- read.csv("foo1.csv",header=T,sep=",")
attach(returnsdata)
foo1 = as.timeSeries(returnsdata)
foo1bind <- returns(window(foo1, start=2007-01-01, end=2007-12-31))

returnsdata <- read.csv("foo2.csv",header=T,sep=",")
attach(returnsdata)
foo2 = as.timeSeries(returnsdata)
foo2bind <- returns(window(foo2, start=2007-01-01, end=2007-12-31))

ralls <- cbind(foo1bind,foo2bind)
colnames(ralls) <- c("FOO1","FOO2")

And then I would like to have the last 1000 lines of this matrix for some computations

index <- ... #some value
past1000 <- ralls[(index-1000):(index-1)]

But

print(past1000)

shows that past1000 doesn't contain either the column of dates, nor the header line, only the columns of returns

Also

print(is.timeSeries(past1000))

returns FALSE, and although I attempted several conversions on past1000, which resulted this check to be TRUE, the effective content of past1000 didn't seem to be changed, still no dates were printed

And despite

print(is.timeSeries(ralls))

returns TRUE, and printing the whole ralls object contains the dates column and the header

print(ralls at .Data[,1])

returns only the second column of ralls (that is, the first column of returns) and no date column is displayed. This seems to indicate that perhaps there is already some problem with ralls, rooted possibly in the datafile reading and binding?

The expectation would be to have past1000 an object where the first column contains the dates, and the remaining columns contain the returns, but I couldn't figure it out so far.

Any suggestion is very much appreciated.

Best regards,
/Johan

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Yohan Chalabi

Wed, Dec 3, 2008 9:30 AM #

JK> Dear All,
   JK> 
   JK> I met some problems with data handling when using the timeSeries library
   JK> 
   JK> I have some csv files, each containing two columns, first column are the series of dates, second are the series prices 
   JK> 
   JK> I read in all files using, and create a matrix with the dates in the first column, and the calculated returns in the remaining columns
   JK> 
   JK> > library(timeSeries)
   JK> 
   JK> > returnsdata <- read.csv("foo1.csv",header=T,sep=",")
   JK> > attach(returnsdata)
   JK> > foo1 = as.timeSeries(returnsdata)
   JK> > foo1bind <- returns(window(foo1, start=2007-01-01, end=2007-12-31))
   JK> 
   JK> > returnsdata <- read.csv("foo2.csv",header=T,sep=",")
   JK> > attach(returnsdata)
   JK> > foo2 = as.timeSeries(returnsdata)
   JK> > foo2bind <- returns(window(foo2, start=2007-01-01, end=2007-12-31))
   JK> 
   JK> > ralls <- cbind(foo1bind,foo2bind)
   JK> > colnames(ralls) <- c("FOO1","FOO2")  
   JK> 
   JK> And then I would like to have the last 1000 lines of this matrix for some computations
   JK> 
   JK> > index <- ... #some value
   JK> > past1000 <- ralls[(index-1000):(index-1)]

you are missing the comma ! This is why you get back a vector. This is
the same behaviour as with a matrix. Without a comma you do not know if
you subset in the rows and columns.

I suppose you wanted to use

ralls[(index-1000):(index-1),]

Note you could use the function tail()

   JK>            
   JK> But 
   JK> > print(past1000) 
   JK> shows that past1000 doesn't contain either the column of dates, nor the header line, only the columns of returns  
   JK> 
   JK> Also 
   JK> > print(is.timeSeries(past1000))
   JK> returns FALSE, and although I attempted several conversions on past1000, which resulted this check to be TRUE, the effective content of past1000 didn't seem to be changed, still no dates were printed
   JK> 
   JK> And despite
   JK> > print(is.timeSeries(ralls))
   JK> returns TRUE, and printing the whole ralls object contains the dates column and the header
   JK> > print(ralls at .Data[,1])           
   JK> returns only the second column of ralls (that is, the first column of returns) and no date column is displayed. This seems to indicate that perhaps there is already some problem with ralls, rooted possibly in the datafile reading and binding?           
   JK> 
   JK> The expectation would be to have past1000 an object where the first column contains the dates, and the remaining columns contain the returns, but I couldn't figure it out so far.
   JK> 
   JK> Any suggestion is very much appreciated.
   JK> 
   JK> Best regards,
   JK> /Johan
   JK> 
   JK> _______________________________________________
   JK> R-SIG-Finance at stat.math.ethz.ch mailing list
   JK> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
   JK> -- Subscriber-posting only.
   JK> -- If you want to post, subscribe first.

PhD student
Swiss Federal Institute of Technology
Zurich

www.ethz.ch

sprohl at na.uni-tuebingen.de

Fri, Dec 5, 2008 3:10 AM #

Dear All,
I have some problems with downloading and matching daily data
of irregular frequency. I tried to apply zoo-codes to read file

data<-read.zoo("C://data.txt",sep="",format="%Y %b %d")
and get the following message:

Fehler in scan(file, what, nmax, sep, dec, quote, skip, nlines,
na.strings,  :  Zeile 31 hatte keine 2 Elemente

The data has missing values. How could I properly download them as
time series and match different datasets by "Date". Unfrtunately,
conventional procedures do not work properly.
Thanks in advance,
Soeren

"JK" == J Konnonen <jkonn2001 at yahoo.com>
on Wed, 3 Dec 2008 06:45:45 -0800 (PST)

   JK> Dear All,
   JK>
   JK> I met some problems with data handling when using the timeSeries
library
   JK>
   JK> I have some csv files, each containing two columns, first column
are the series of dates, second are the series prices
   JK>
   JK> I read in all files using, and create a matrix with the dates in
the first column, and the calculated returns in the remaining columns
   JK>
   JK> > library(timeSeries)
   JK>
   JK> > returnsdata <- read.csv("foo1.csv",header=T,sep=",")
   JK> > attach(returnsdata)
   JK> > foo1 = as.timeSeries(returnsdata)
   JK> > foo1bind <- returns(window(foo1, start=2007-01-01,
end=2007-12-31))
   JK>
   JK> > returnsdata <- read.csv("foo2.csv",header=T,sep=",")
   JK> > attach(returnsdata)
   JK> > foo2 = as.timeSeries(returnsdata)
   JK> > foo2bind <- returns(window(foo2, start=2007-01-01,
end=2007-12-31))
   JK>
   JK> > ralls <- cbind(foo1bind,foo2bind)
   JK> > colnames(ralls) <- c("FOO1","FOO2")
   JK>
   JK> And then I would like to have the last 1000 lines of this matrix
for some computations
   JK>
   JK> > index <- ... #some value
   JK> > past1000 <- ralls[(index-1000):(index-1)]

you are missing the comma ! This is why you get back a vector. This is
the same behaviour as with a matrix. Without a comma you do not know if
you subset in the rows and columns.

I suppose you wanted to use

ralls[(index-1000):(index-1),]

Note you could use the function tail()

   JK>
   JK> But
   JK> > print(past1000)
   JK> shows that past1000 doesn't contain either the column of dates, nor
the header line, only the columns of returns
   JK>
   JK> Also
   JK> > print(is.timeSeries(past1000))
   JK> returns FALSE, and although I attempted several conversions on
past1000, which resulted this check to be TRUE, the effective content
of past1000 didn't seem to be changed, still no dates were printed
   JK>
   JK> And despite
   JK> > print(is.timeSeries(ralls))
   JK> returns TRUE, and printing the whole ralls object contains the
dates column and the header
   JK> > print(ralls at .Data[,1])
   JK> returns only the second column of ralls (that is, the first column
of returns) and no date column is displayed. This seems to indicate
that perhaps there is already some problem with ralls, rooted possibly
in the datafile reading and binding?
   JK>
   JK> The expectation would be to have past1000 an object where the first
column contains the dates, and the remaining columns contain the
returns, but I couldn't figure it out so far.
   JK>
   JK> Any suggestion is very much appreciated.
   JK>
   JK> Best regards,
   JK> /Johan
   JK>
   JK> _______________________________________________
   JK> R-SIG-Finance at stat.math.ethz.ch mailing list
   JK> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
   JK> -- Subscriber-posting only.
   JK> -- If you want to post, subscribe first.


--
PhD student
Swiss Federal Institute of Technology
Zurich

www.ethz.ch

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Brian G. Peterson

Fri, Dec 5, 2008 4:03 AM #

Please attach a short sample of your data file that replicates your error.

sprohl at na.uni-tuebingen.de wrote:

Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

sprohl at na.uni-tuebingen.de

Fri, Dec 5, 2008 5:50 AM #

Hello Brian,
please find attached sample of the data
data[1:5,]
  X2002.07.01 X0.0071
1  2002-07-02  0.0072
2  2002-07-03  0.0072
3  2002-07-04  0.0073
4  2002-07-05  0.0073
5  2002-07-08  0.0073
str(data)
'data.frame':   1503 obs. of  2 variables:
 $ X2002.07.01: Factor w/ 1503 levels "2002-07-02","2002-07-03",..: 1 2 3
4 5 6 7 8 9 10 ...
 $ X0.0071    : num  0.0072 0.0072 0.0073 0.0073 0.0073 0.0073 0.0073
0.0073 0.0073 0.0074 ...
datas<-as.xts(data)
Fehler in as.POSIXlt.character(x, tz, ...) :
character string is not in a standard unambiguous format

How could I convert the data to xts?
Thank in advance,
Soeren





Please attach a short sample of your data file that replicates your error.

Joshua Ulrich

Fri, Dec 5, 2008 7:29 AM #

Hi Soeren,

It appears the function you used to read your data into R expected the
file to contain a header row.  You should use the equivalent of
read.csv(file, header=FALSE).

as.xts() works on data.frames that have rownames.  The default is to
convert the rownames to POSIXct.

I recommend you try:
datas <- xts(data[,-1],as.POSIXct(data[,1]))

Best,
Josh
--
http://quantemplation.blogspot.com

On Fri, Dec 5, 2008 at 7:50 AM, <sprohl at na.uni-tuebingen.de> wrote:

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

sprohl at na.uni-tuebingen.de

Fri, Dec 5, 2008 7:35 AM #

thanks for this suggetsion. I tried to read the data in this way and
to convert in xts, and get the following message
data <- xts(data[,-1],as.POSIXct(data[,1]))
Fehler in as.POSIXlt.character(as.character(x)) :
character string is not in a standard unambiguous format

Do you know what is going wrong?
Thanks in advance,
Best,
Soeren



Hi Soeren,

It appears the function you used to read your data into R expected the
file to contain a header row.  You should use the equivalent of
read.csv(file, header=FALSE).

as.xts() works on data.frames that have rownames.  The default is to
convert the rownames to POSIXct.

I recommend you try:
datas <- xts(data[,-1],as.POSIXct(data[,1]))

Best,
Josh
--
http://quantemplation.blogspot.com



On Fri, Dec 5, 2008 at 7:50 AM,  <sprohl at na.uni-tuebingen.de> wrote:

Hello Brian,
please find attached sample of the data
data[1:5,]
 X2002.07.01 X0.0071
1  2002-07-02  0.0072
2  2002-07-03  0.0072
3  2002-07-04  0.0073
4  2002-07-05  0.0073
5  2002-07-08  0.0073
str(data)
'data.frame':   1503 obs. of  2 variables:
 $ X2002.07.01: Factor w/ 1503 levels "2002-07-02","2002-07-03",..: 1 2
3
4 5 6 7 8 9 10 ...
 $ X0.0071    : num  0.0072 0.0072 0.0073 0.0073 0.0073 0.0073 0.0073
0.0073 0.0073 0.0074 ...
datas<-as.xts(data)
Fehler in as.POSIXlt.character(x, tz, ...) :
character string is not in a standard unambiguous format

How could I convert the data to xts?
Thank in advance,
Soeren





Please attach a short sample of your data file that replicates your
error.

sprohl at na.uni-tuebingen.de wrote:

Dear All,
I have some problems with downloading and matching daily data
of irregular frequency. I tried to apply zoo-codes to read file

data<-read.zoo("C://data.txt",sep="",format="%Y %b %d")
and get the following message:

Fehler in scan(file, what, nmax, sep, dec, quote, skip, nlines,
na.strings,  :  Zeile 31 hatte keine 2 Elemente

The data has missing values. How could I properly download them as
time series and match different datasets by "Date". Unfrtunately,
conventional procedures do not work properly.
Thanks in advance,
Soeren


--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.