What is the most cost effective hardware for R?
On Tue, 8 May 2012, Hugh Morgan wrote:
Perhaps I have confused the issue. When I initially said "data points" I meant one stand alone analysis, not one piece of data. Each analysis point takes 1.5 seconds. I have not implemented running this over the whole dataset yet, but I would expect it to take about 5 to 10 hours. This is just about acceptable, but it would be better if this was quicker. As I say, the exact analysis method has not yet been determined, and if that was significantly more computationally intensive then that could be an issue.
If I had to do what you write above, I would separate the data into chunks; one for each core/CPU in my system. Then I would invoke R to run on each core/CPU and have that instance process one data set. With sufficient memory for each core/CPU the processing will occur in parallel and cut the overall time by the number of instances running. You might want to turn up the air conditioning around the system 'cause that CPU is going to be working hard. Rich