Google data download with getSymbols - is there a limit/quota ?
On 03/23/2013 04:14 AM, me wrote:
I am trying to use getSymbols(...,src='google') from the quantmod package for downloading google finance data for about 1600 symbols, once a week. Some of that will be used in several quantstrat strategies. I am seeing that after having gotten about 500 symbols, trying to download more will produce http errors. If I re-try those symbols the next day, then the quota seems to have been reset and I can download around 500 symbols again. Question: Is anyone else experiencing this ? Would anyone know what is the minimum time I have to wait before I can download the next batch of 500 symbols (I hope it's somewhere around several minutes or several hours, but not more than that) ?
getSymbols has for years included a pause between requests for multiple symbols, to try to limit the impact you have on their infrastructure, and avoid some throttling. I suspect that if you space out your requests, you'll naturally hit different servers, and probably be fine, but I can't speak from personal experience. Obviously, if you intend to use this data for investment decisions, you're likely violating the Google or Yahoo terms of use anyway, and you'd likely be better off getting the data from a broker source such as InteractiveBrokers. Regards, - Brian
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock