Skip to content

dataset to xts conversion issue with timeseries data

7 messages · Ilya Kipnis, Aritra Pan, Brian G. Peterson +1 more

Original
Aritra Pan
# 
Sorry, if it is becoming duplicate message. Any suggestion will be highly appreciated. 

-- 
Thanks, 
Aritra 

----- Original Message -----

From: "Aritra Pan" <aritrapan at vgsom.iitkgp.ernet.in> 
To: "R-SIG-Finance" <r-sig-finance-bounces at r-project.org> 
Sent: Thursday, May 12, 2016 11:28:41 AM 
Subject: [R-SIG-Finance] dataset to xts conversion issue with timeseries data 

Hello Members, 

I am hoping if I can get few suggestions on the issue I am facing. 

I am taking data from server using Rblpapi package for 40-50 securities. The data looks like below: 

times 	askValue 	askSize 	stockName 	Type 
01/02/2016 9:15 	91.8 	500 	ABCD IS Equity 	Ask 
01/02/2016 9:15 	92.4 	1420 	ABCD IS Equity 	Ask 
01/02/2016 9:15 	92.2 	5000 	ABCD IS Equity 	Ask 
01/02/2016 9:15 	91.9 	6524 	ABCD IS Equity 	Ask 
01/02/2016 9:15 	91.9 	11524 	ABCD IS Equity 	Ask 
01/02/2016 9:15 	92.05 	6824 	ABCD IS Equity 	Ask 

ABCD IS - any security (assume). 

Now this data is coming in standard dataset format to R from data server. When I try to convert it to XTS it becomes like below 

	askValue 	askSize 	stockName 	askType 
01/02/2016 9:15 	" 91.80" 	" 500" 	"ABCD IS Equity" 	"Ask" 
01/02/2016 9:15 	"474.45" 	" 300" 	"EFGH IS Equity" 	"Ask" 
01/02/2016 9:15 	"455.00" 	" 100" 	"XYZW IS Equity" 	"Ask" 
01/02/2016 9:15 	"101.50" 	" 2799" 	"MNP IS Equity" 	"Ask" 
01/02/2016 9:15 	"194.95" 	" 1600" 	"IJK IS Equity" 	"Ask" 
01/02/2016 9:15 	"293.45" 	" 13" 	"PQRS IS Equity" 	"Ask" 

You can see the the column name "times" is becoming here index. And that's why after data conversion to xts object I am not able to match my quote data which are bid dataset and ask dataset. 

Can anyone please suggest on this data set to xts conversion. 

PS: When I am downloading data in excel and then transforming data from excel to xts then it is working fine. 

	Date 	Type 	Price 	Size 
1 	01/03/2016 9:15 	ASK 	538.9 	50 
2 	01/03/2016 9:15 	ASK 	538.8 	75 
3 	01/03/2016 9:15 	ASK 	538.7 	50 
4 	01/03/2016 9:15 	ASK 	538.3 	75 
5 	01/03/2016 9:15 	ASK 	538.85 	798 
6 	01/03/2016 9:15 	ASK 	538.9 	93 

But as I am trying to run for multiple stocks data download script at same time and it is high-frequency data, so I am using R script to download data from server to R directly skipping excel using Rblpapi package.
Ilya Kipnis
# 
Aritra,

The underlying data structure of an xts is a matrix. If you have a
character vector inside that matrix, the entire thing will become
characters. In your example, you have stockName and type vectors, which are
character vectors, whereas in the excel example, you don't. Hope that helps.

-Ilya

On Fri, May 13, 2016 at 12:31 AM, Aritra Pan <
aritrapan at vgsom.iitkgp.ernet.in> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Aritra Pan
    

    

      
      #
    

  


  

      
Hi Ilya, 

Thanks for the suggestion. But when I try with the dataset containing only timestamp, value and size columns; then also if I try to convert from dataset to xts, the timestamp column is becoming index and further I am not able to access timestamp matching between objects. 

-- 
Thanks, 
Aritra 

----- Original Message -----

From: "Ilya Kipnis" <ilya.kipnis at gmail.com> 
To: "Aritra Pan" <aritrapan at vgsom.iitkgp.ernet.in> 
Cc: r-sig-finance at r-project.org 
Sent: Friday, May 13, 2016 10:39:18 AM 
Subject: Re: [R-SIG-Finance] Fwd: dataset to xts conversion issue with timeseries data 

Aritra, 

The underlying data structure of an xts is a matrix. If you have a character vector inside that matrix, the entire thing will become characters. In your example, you have stockName and type vectors, which are character vectors, whereas in the excel example, you don't. Hope that helps. 

-Ilya

        
On Fri, May 13, 2016 at 12:31 AM, Aritra Pan < aritrapan at vgsom.iitkgp.ernet.in > wrote:

        
Sorry, if it is becoming duplicate message. Any suggestion will be highly appreciated. 

-- 
Thanks, 
Aritra 

----- Original Message ----- 

From: "Aritra Pan" < aritrapan at vgsom.iitkgp.ernet.in > 
To: "R-SIG-Finance" < r-sig-finance-bounces at r-project.org > 
Sent: Thursday, May 12, 2016 11:28:41 AM 
Subject: [R-SIG-Finance] dataset to xts conversion issue with timeseries data 

Hello Members, 

I am hoping if I can get few suggestions on the issue I am facing. 

I am taking data from server using Rblpapi package for 40-50 securities. The data looks like below: 

times askValue askSize stockName Type 
01/02/2016 9:15 91.8 500 ABCD IS Equity Ask 
01/02/2016 9:15 92.4 1420 ABCD IS Equity Ask 
01/02/2016 9:15 92.2 5000 ABCD IS Equity Ask 
01/02/2016 9:15 91.9 6524 ABCD IS Equity Ask 
01/02/2016 9:15 91.9 11524 ABCD IS Equity Ask 
01/02/2016 9:15 92.05 6824 ABCD IS Equity Ask 

ABCD IS - any security (assume). 

Now this data is coming in standard dataset format to R from data server. When I try to convert it to XTS it becomes like below 

askValue askSize stockName askType 
01/02/2016 9:15 " 91.80" " 500" "ABCD IS Equity" "Ask" 
01/02/2016 9:15 "474.45" " 300" "EFGH IS Equity" "Ask" 
01/02/2016 9:15 "455.00" " 100" "XYZW IS Equity" "Ask" 
01/02/2016 9:15 "101.50" " 2799" "MNP IS Equity" "Ask" 
01/02/2016 9:15 "194.95" " 1600" "IJK IS Equity" "Ask" 
01/02/2016 9:15 "293.45" " 13" "PQRS IS Equity" "Ask" 

You can see the the column name "times" is becoming here index. And that's why after data conversion to xts object I am not able to match my quote data which are bid dataset and ask dataset. 

Can anyone please suggest on this data set to xts conversion. 

PS: When I am downloading data in excel and then transforming data from excel to xts then it is working fine. 

Date Type Price Size 
1 01/03/2016 9:15 ASK 538.9 50 
2 01/03/2016 9:15 ASK 538.8 75 
3 01/03/2016 9:15 ASK 538.7 50 
4 01/03/2016 9:15 ASK 538.3 75 
5 01/03/2016 9:15 ASK 538.85 798 
6 01/03/2016 9:15 ASK 538.9 93 

But as I am trying to run for multiple stocks data download script at same time and it is high-frequency data, so I am using R script to download data from server to R directly skipping excel using Rblpapi package.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Brian G. Peterson
    

    

      
      #
    

  


  

      
Please make it easy for someone else to help you.  Provide a minimal 
example, e.g. use dput to export a small copy of your object showing us 
what you're trying to do and the code you are trying.

Ilya's advice was correct, so your problem is unquestionably user error, 
but it will be hard for anyone to help you if you don't show us 
precisely what you are trying by providing a reproducible example 
including a copy of your input data.

        
On 05/13/2016 12:17 AM, Aritra Pan wrote:

            

      
      
      
    

        

        
On 05/13/2016 12:09 AM, Ilya Kipnis wrote:

            

      
      
      
    

        
which are

            

      
      
      
    

        
helps.

            

      
      
      
    

        
that's

            

      
      
      
    

        
same

            

      
      
      
    

        
download data

            

      
      
      
    

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Aritra Pan
    

    

      
      #
    

  


  

      
Hi,

Sorry if my mail lacked information. But I took Ilya's point and it's resolved now. Still I will explain the issue once more.

            

      
      
      
    

        
times askValue askSize      stockName askType
1 2016-02-01 09:15:01    91.80     500 DITV IS Equity     Ask
2 2016-02-01 09:15:02    92.40    1420 DITV IS Equity     Ask
3 2016-02-01 09:15:04    92.20    5000 DITV IS Equity     Ask
4 2016-02-01 09:15:05    91.90    6524 DITV IS Equity     Ask
5 2016-02-01 09:15:06    91.90   11524 DITV IS Equity     Ask
6 2016-02-01 09:15:06    92.05    6824 DITV IS Equity     Ask

This is the data from data set. Now if I convert it to xts then the "times" column is converting to index (as for time series data). So to select data from xts matching particular timestamp was becoming difficult. That's what Ilya explained that matrix data structure of xts.

-- 
Thanks, 
Aritra 

----- Original Message -----
From: "Brian G. Peterson" <brian at braverock.com>
To: r-sig-finance at r-project.org
Sent: Friday, May 13, 2016 3:32:04 PM
Subject: Re: [R-SIG-Finance] Fwd: dataset to xts conversion issue with timeseries data

Please make it easy for someone else to help you.  Provide a minimal 
example, e.g. use dput to export a small copy of your object showing us 
what you're trying to do and the code you are trying.

Ilya's advice was correct, so your problem is unquestionably user error, 
but it will be hard for anyone to help you if you don't show us 
precisely what you are trying by providing a reproducible example 
including a copy of your input data.

        
On 05/13/2016 12:17 AM, Aritra Pan wrote:

            

      
      
      
    

        

        
On 05/13/2016 12:09 AM, Ilya Kipnis wrote:

            

      
      
      
    

        
which are

            

      
      
      
    

        
helps.

            

      
      
      
    

        
that's

            

      
      
      
    

        
same

            

      
      
      
    

        
download data

            

      
      
      
    

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Joshua Ulrich
    

    

      
      #
    

  


  

      
You still have not explained what you're trying to do.  You say, "to
select data from xts matching particular timestamp was becoming
difficult."  Selecting data for particular timestamps is one of the
problems xts was *designed* to solve, so I doubt xts makes it
difficult.

In your initial email, you said, "I am not able to match my quote data
which are bid dataset and ask dataset."  You need to provide an
example of your quote data, and an example of your desired result from
matching them.

It is much better to describe your problem using raw data and code
than by using words, especially when there's a language barrier.  It
appears English isn't your native language, and many on this list
probably do not speak your native language, but we all speak R.


On Fri, May 13, 2016 at 8:29 AM, Aritra Pan

        
<aritrapan at vgsom.iitkgp.ernet.in> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Aritra Pan
    

    

      
      #
    

  


  

      
Hi Joshua,

1. If I could share original data with forum, I would love to. But there are some constraints. So, I pasted masked data in my first mail.
2. I somehow missed the matrix structure of xts objects. First, I was hoping to keep the structure from dataframe intact in xts. Ilya cleared the concept.

Hope these resolves your concern.

Enjoy your weekend,

PS: Sorry if I have used bad English.

-- 
Thanks, 
Aritra 

----- Original Message -----
From: "Joshua Ulrich" <josh.m.ulrich at gmail.com>
To: "Aritra Pan" <aritrapan at vgsom.iitkgp.ernet.in>
Cc: "Brian G. Peterson" <brian at braverock.com>, "r-sig-finance" <r-sig-finance at r-project.org>
Sent: Friday, May 13, 2016 7:41:05 PM
Subject: Re: [R-SIG-Finance] Fwd: dataset to xts conversion issue with timeseries data

You still have not explained what you're trying to do.  You say, "to
select data from xts matching particular timestamp was becoming
difficult."  Selecting data for particular timestamps is one of the
problems xts was *designed* to solve, so I doubt xts makes it
difficult.

In your initial email, you said, "I am not able to match my quote data
which are bid dataset and ask dataset."  You need to provide an
example of your quote data, and an example of your desired result from
matching them.

It is much better to describe your problem using raw data and code
than by using words, especially when there's a language barrier.  It
appears English isn't your native language, and many on this list
probably do not speak your native language, but we all speak R.


On Fri, May 13, 2016 at 8:29 AM, Aritra Pan

        
<aritrapan at vgsom.iitkgp.ernet.in> wrote: