Retrieving latest day's data

Hi,

I realise that this isn't a 100% R-SIG question and is probably a more general R question, but I think it is related, and when I sort it it is leading to an R-SIG question, so if I could ask you to bear with me.

I'm trying to build my own trading model, to test out some strategies.? Part of this involves downloading the stcok market data and then processing it, i.e. running my model on it, using R-SIG to build graphs etc.? However using Yahoo the historical data is always one day behind.? To get around this I'm attempting to use a screen scraper utility, called PageScrape.? It's command line based, however I'm trying to use system() to call the screen scraper from within R.? The problem is that the screen scraper use regex and this causes unexpected symbol errors when I try to run it inside the system function.

So, I can run the below inside a batch file, so can you if you download PageScraper:
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+) - [0-9]+\.[0-9]+</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e">Last Trade:</td><td class=.yfnc_tabledata1.><big><b>([0-9]+\.[0-9]+) p"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Open:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Volume:</td><td class=.yfnc_tabledata1.>([0-9]+,[0-9]+,[0-9]+)</td></tr>"
pause

This is actually 4 lines, which will bring back the open, high, low and volume from the URL after -u.? The bit inside the () is what's returned.

What I'm after is to convert this to:

system('cmd /c "pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>""')

I've tried several attempts to get it working without any success.? If anyone here can help that would be terrific, and hopefully some of you may find it useful.

Many thanks,
Martin.




? ? ? ?[[alternative HTML version deleted]]

Hi,

I realise that this isn't a 100% R-SIG question and is probably a more general R question, but I think it is related, and when I sort it it is leading to an R-SIG question, so if I could ask you to bear with me.

I'm trying to build my own trading model, to test out some strategies.  Part of this involves downloading the stcok market data and then processing it, i.e. running my model on it, using R-SIG to build graphs etc.  However using Yahoo the historical data is always one day behind.  To get around this I'm attempting to use a screen scraper utility, called PageScrape.  It's command line based, however I'm trying to use system() to call the screen scraper from within R.  The problem is that the screen scraper use regex and this causes unexpected symbol errors when I try to run it inside the system function.

So, I can run the below inside a batch file, so can you if you download PageScraper:
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+) - [0-9]+\.[0-9]+</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e">Last Trade:</td><td class=.yfnc_tabledata1.><big><b>([0-9]+\.[0-9]+) p"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Open:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Volume:</td><td class=.yfnc_tabledata1.>([0-9]+,[0-9]+,[0-9]+)</td></tr>"
pause

This is actually 4 lines, which will bring back the open, high, low and volume from the URL after -u.  The bit inside the () is what's returned.

What I'm after is to convert this to:

system('cmd /c "pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>""')

I've tried several attempts to get it working without any success.  If anyone here can help that would be terrific, and hopefully some of you may find it useful.

Many thanks,
Martin.



      
	[[alternative HTML version deleted]]

  
------------------------------------------------------------------------

getSymbols("AAPL")

head(index(AAPL))

head(time(AAPL))

write.zoo(tail(AAPL))

Hi,

Thanks for the suggestions, just what I need, however just one more thing...

When I use:

getSymbols("AML.L", src="google")
x <- tail(AML.L,1)

I get back:

?????????? AML.L.Open AML.L.High AML.L.Low AML.L.Close AML.L.Volume
2009-11-17????? 385.5????? 388.9???? 381.6?????? 382.6????? 1176883

All the variables above are named, apart from the date.? What I would like to do is have an if x$AML.L.Date = Sys.Date() then update my text file.? This means that I can update the historical data that I download from Yahoo in the evening, run my model and look for any buys, so I can save a day and do my research in the evening before the next morning.? Unfortunately for some reason I can't access the date, high low close open and volume are there but not date.

Any help please?

Thanks again,
Martin.



--- On Tue, 11/17/09, Cedrick W. Johnson <cedrick at cedrickjohnson.com> wrote:

From: Cedrick W. Johnson <cedrick at cedrickjohnson.com>
Subject: Re: [R-SIG-Finance] ?Retrieving latest day's data
To: "Martin Jenkins" <mjenko at yahoo.com>, "R-SIG-Finance" <r-sig-finance at stat.math.ethz.ch>
Date: Tuesday, November 17, 2009, 2:52 AM

I have found that typically after the evening extended sessions finish (~8PM ET) I can get today's closing prices. I just ran this now @ 9:45P et and got my closing prices for today:

? ? ? ? ? GSPC.Open GSPC.High GSPC.Low GSPC.Close GSPC.Volume GSPC.Adjusted
2009-11-13???1087.59???1097.79? 1085.33? ? 1093.48? 3792610000? ? ???1093.48
2009-11-16???1094.13???1113.69? 1094.13? ? 1109.30? 4565850000? ? ???1109.30

I see you're getting UK tickers (this is what Joshua suggested, the getQuote function in quantmod):

getQuote("AML.L")

? ? ? ? ? ? ? Trade Time? Last Change % Change Open? High???Low? Volume
AML.L 2009-11-16 11:35:00 387.4???10.8???+2.87%? 385 395.1 375.6 3375783

*edit*

While composing, I thought to check using Google:

getSymbols("AML.L", src="google")

tail(AML.L,2)

? ? ? ? ? AML.L.Open AML.L.High AML.L.Low AML.L.Close AML.L.Volume
2009-11-13? ? ? 372.9? ? ? 376.9? ???368.1? ? ???376.6? ? ? 1403827
2009-11-16? ? ? 385.0? ? ? 395.1? ???375.6? ? ???387.4? ? ? 3375783

Try that on the rest of your symbols to see what you get. Again, the time(s) that they update the closing price may be different. The same query I just did using Yahoo as a source yielded the 13th's closing price, while google is getting you what you want.

HTH,
Cedrick


Martin Jenkins wrote:

Hi,

I realise that this isn't a 100% R-SIG question and is probably a more general R question, but I think it is related, and when I sort it it is leading to an R-SIG question, so if I could ask you to bear with me.

I'm trying to build my own trading model, to test out some strategies.? Part of this involves downloading the stcok market data and then processing it, i.e. running my model on it, using R-SIG to build graphs etc.? However using Yahoo the historical data is always one day behind.? To get around this I'm attempting to use a screen scraper utility, called PageScrape.? It's command line based, however I'm trying to use system() to call the screen scraper from within R.? The problem is that the screen scraper use regex and this causes unexpected symbol errors when I try to run it inside the system function.

So, I can run the below inside a batch file, so can you if you download PageScraper:
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+) - [0-9]+\.[0-9]+</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e">Last Trade:</td><td class=.yfnc_tabledata1.><big><b>([0-9]+\.[0-9]+) p"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Open:</td><td class=.yfnc_tabledata1.>([0-9]+\.[0-9]+)</td></tr>"
pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Volume:</td><td class=.yfnc_tabledata1.>([0-9]+,[0-9]+,[0-9]+)</td></tr>"
pause

This is actually 4 lines, which will bring back the open, high, low and volume from the URL after -u.? The bit inside the () is what's returned.

What I'm after is to convert this to:

system('cmd /c "pscrape -u"http://uk.finance.yahoo.com/q/ta?s=AML.L&t=1y&l=off&z=l&q=b&p=&a=&c=" -e"Day's Range:</td><td class=.yfnc_tabledata1.>[0-9]+\.[0-9]+ - ([0-9]+\.[0-9]+)</td></tr>""')

I've tried several attempts to get it working without any success.? If anyone here can help that would be terrific, and hopefully some of you may find it useful.

Many thanks,
Martin.



? ? ?????? [[alternative HTML version deleted]]

???------------------------------------------------------------------------

? ? ? ?[[alternative HTML version deleted]]