Running R as a server or in a cluster
Yes - its JBOC (just a bunch of computers). You provide them with a disk image (of sorts) and they will load it on to as many computers as you request. Images are loaded and machines are requested via a web services API. Initially you can request up to 20 machines - but if you email them you can ask for more. All network bandwidth between machines is free, but there is a per GB transfer charge for external connectivity - I can't recall what the rate is, but it is very reasonable. Not being a specialized grid environment, all inter-node communication and scheduling has to be handled by your own application. But for the price, that's not too bad. While I was aware of SNOW, I'm not familiar with the other clustering approaches mentioned earlier in this thread. What special sauce does Sun provide to make running on a grid easier than running on a JBOC style setup? Josh -----Original Message----- From: Jeff Ryan [mailto:jeff.a.ryan at gmail.com] Sent: Wednesday, September 26, 2007 10:18 AM To: Joshua Reich Cc: Brian G. Peterson; r-sig-finance at stat.math.ethz.ch Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster I do know the Sun one is using their grid software, and is supposedly highly secure. Basically have access to a 2000 node opteron cluster. The Amazon one seems to be more of using a machine, one at a time. Is that correct?
On 9/26/07, Joshua Reich <josh at gghc.com> wrote:
We recently set up a similar environment using Amazon's EC2 service. They charge $0.1 per CPU hour. I can't say what our results have been like yet - still ironing out the kinks in our R code. But I will certainly let you all know how it goes. Our 'clustering' mechanism is very simple. We have written perl scripts that receive data over HTTP, start R, process the data, and then post the results back via HTTP to a central server. Josh -----Original Message----- From: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of Jeff Ryan Sent: Wednesday, September 26, 2007 10:08 AM To: Brian G. Peterson Cc: r-sig-finance at stat.math.ethz.ch Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster Hi all, Short of answers, but I do wonder if anyone has used Sun Microsystems www.network.com for grid work with R. At 1USD a CPU hr, with R already built - and a working example script on the service - it seems
like a path worth exploring. Has anyone given it a try. I set up an account, but have yet to get the opportunity to try it out. Here is the link: http://www.network.com/apps/r_project.html Jeff Ryan On 9/26/07, Brian G. Peterson <brian at braverock.com> wrote:
Adrian Dragulescu wrote:
We have set up a Condor cluster, see http://www.cs.wisc.edu/condor/ and we submit R jobs to the cluster. It works well because Condor has very advanced
scheduling capabilities, job monitoring, etc.
Adrian, Could you provide more details? Are you running Rserve on the cluster, running "R CMD BATCH", or using Parallel-R? I'd like to suggest that we use this thread to continue to develop the
collective knowledge of the r-sig-finance community on distributed
or high-throughput R calculations.
Regards,
- Brian
_______________________________________________ R-SIG-Finance at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. -- If you want to post, subscribe first.
_______________________________________________ R-SIG-Finance at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. -- If you want to post, subscribe first.