An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110513/d15f8771/attachment.pl>
Powerful PC to run R
12 messages · Michael Haenlein, Mike Marchywka, Hrishi Mittal +9 more
Date: Fri, 13 May 2011 12:38:51 +0200 From: haenlein at escpeurope.eu To: r-help at r-project.org Subject: [R] Powerful PC to run R Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating. I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance.
( I think my laptop is overheating with firefox trying to execute whatever stupid code hotmail is using? ssh to remote server echos keys faster LOL) The point of the above is that it really depends what you are doing. Heat can com from disk drive as well as silicon. Generally you'd want to consider algorithm and implementation and get profiling info before just buying bigger hammer. If you are thrashing VM, sorting some data may help for example. If it is suited parallelism, you could even try to distribute task over several cheaper computers, hard to knwo.
Thanks,
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110513/a534c3ab/attachment.pl>
On Fri, May 13, 2011 at 11:38 AM, Michael Haenlein
<haenlein at escpeurope.eu> wrote:
I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance.
Michael Haenlein Assocaite Professor of Marketing
As a Professor of Marketing surely you buy whatever Apple tell you to? Seriously, 'as fast as possible'? No financial constraints? Then spend several billion in a factory and some hardware developers to run R on bare silicon. Too much? Spend a million on a data centre and stuff it full of rack servers, and some software developers to make your algorithms run in parallel on the cluster. Secondly, speed is massively dependent on exactly what you are doing. Some jobs are I/O-bound, they can only go as fast as they can read in or write out data. Some are limited by available RAM, and start swapping bits of memory to hard disk, slowing things up. Some are CPU-bound and can go faster by plugging in a faster processor. Some are bound by internal bus speeds, and can't shuttle information between RAM and CPU fast enough. The solution to each of these problems is different. For example, there's no point in buying an 8-core CPU if your programs can only use 1 core at a time, and you don't think you'll be running 8 programs at once. Conclusion: for a desktop PC, get as much fast RAM, the fastest CPU and the quickest HD you can find. That should cover all the possible bottlenecks. Barry
Given the price of the Thinkpad X201 I would be very upset if it overheated and would contact my supplier for a replacement. I also understand that this CPU may slow down if it is overheated. You would probably get a better performance from a desktop workstation or a larger laptop with better ventilation. As already stated a large mumer of cpu's and multithreading are of little advantage when you are running standard R. John
On 13 May 2011 11:38, Michael Haenlein <haenlein at escpeurope.eu> wrote:
Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating. I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance. Thanks, Michael Michael Haenlein Assocaite Professor of Marketing ESCP Europe Paris, France ? ? ? ?[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
John C Frain Economics Department Trinity College Dublin Dublin 2 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj at tcd.ie mailto:frainj at gmail.com
2 days later
On Fri, May 13, 2011 at 6:38 AM, Michael Haenlein
<haenlein at escpeurope.eu> wrote:
I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
If you are on Windows press the Windows key and type in Power Options. When the associated dialog pops up choose Power Saver. Now your PC will use less power so it won't heat up so much although your performance could suffer a bit. Also ensure that there is sufficient air circulation around the machine.
Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
Also: A previous post in this tread suggested "Rprof" [sec. 3.2 in
"Writing R Extensions", available via help.start()]. This should
identify the functions that consume the most time. The standard
procedure to improve speed is as follows:
1. Experiment with different ways of computing the same thing in
R. In many cases, this can help you reduce the compute time by a factor
of 10 or even 1,000 or more. Try this, perhaps using proc.time and
system.time with portions of your code, the rerun Rprof.
2. After you feel you have done the best you can with R, you
might try coding the most compute intensive portion of the algorithm in
a compiled language like C, C++ or Fortran. Then rerun Rprof, etc.
3. After trying (or not) compiled code, it may be appropriate to
consider "CRAN Task View: High-Performance and Parallel Computing with
R". (From a CRAN mirror, select "Task Views" ->
"HighPerformanceComputing: High-Performance and Parallel Computing with
R".) You may also want to try the "foreach" package from Revolution
Computing (revolutionanalytics.com). These capabilities can help you
get the most out of a multi-core computer. NOTE: While your code is
running, you can check the "Performance" tab in Windows Task Manager to
see what percent of your CPUs and physical memory you are using. I
mention this, because without "foreach" you might get at most 1 of your
4 CPUs running R. With "foreach", you might be able to get all of them
working for you. Then after you have done this and satisfied yourself
that you've done the best you can with all of this, I suggest you try
the Amazon Cloud.
If you have not already solved your problem with this and have
not yet tried these three steps, I suggest you try this. It may take
more of your time, but you will likely learn much that will help you in
the future as well as help you make a better choice of a new computer if
you ultimately decide to do that.
Hope this helps.
Spencer
On 5/15/2011 8:28 AM, Gabor Grothendieck wrote:
On Fri, May 13, 2011 at 6:38 AM, Michael Haenlein <haenlein at escpeurope.eu> wrote:
I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
If you are on Windows press the Windows key and type in Power Options. When the associated dialog pops up choose Power Saver. Now your PC will use less power so it won't heat up so much although your performance could suffer a bit. Also ensure that there is sufficient air circulation around the machine.
Spencer Graves, PE, PhD President and Chief Operating Officer Structure Inspection and Monitoring, Inc. 751 Emerson Ct. San Jos?, CA 95126 ph: 408-655-4567
On May 13, 2011, at 6:38 AM, Michael Haenlein wrote:
Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
You didn't mention whether you are using a 64-bit OS or not. A single 32-bit process can not use more than 2 GB RAM. If your calculations would benefit from the full 8 GB RAM on your machine, you need to be able to run 64-bit R. My understanding is that, on Windows, you either have to install the OS as 32-bit and use all 32-bit software or install 64-bit Windows and run all 64-bit software. A Mac can run 32-bit and 64-bit software simultaneously and I'm not sure about Linux. In the case of Linux, it probably doesn't matter so much because most Linux software is available as open source and you can compile it yourself either way.
I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance.
You need to evaluate whether RAM or raw processor speed is most critical for what you're doing. In my case, I upgraded my Mac Pro to 16 GB RAM and was able to do hierarchical clustering heatmaps overnight which previously took more than a week to compute. Using the Activity Monitor utility, it looks like some of the, even larger, heatmap computations would benefit from 32 GB RAM or more. Linux runs on the widest range of hardware and that allows you the greatest ability to shop around. If RAM is the deciding factor, then you can look around for a machine which can hold as much RAM as possible. If processor speed is the factor, then you can optimize for that. Windows runs on a reasonable array of hardware but has the disadvantage that the OS, itself, uses a lot of resources. The Mac has the advantage of flexibility. When you download the precompiled R package, it comes with both a 32-bit and a 64-bit executable. This is because 32-bit processes run a little faster if you don't need large amounts of RAM. If you do need the RAM, then you run the 64-bit version. Aram Fingal
On Sun, May 15, 2011 at 9:31 AM, Spencer Graves
<spencer.graves at structuremonitoring.com> wrote:
Also: ?A previous post in this tread suggested "Rprof" [sec. 3.2 in "Writing R Extensions", available via help.start()]. This should identify the functions that consume the most time. ?The standard procedure to improve speed is as follows: ? ? ?1. ?Experiment with different ways of computing the same thing in R. ?In many cases, this can help you reduce the compute time by a factor of 10 or even 1,000 or more. ?Try this, perhaps using proc.time and system.time with portions of your code, the rerun Rprof.
I second this one; if you have things running for weeks, and you haven't done any serious optimization already, you most likely can bring that down to days or hours by investigating where the bottlenecks are. Here is a good illustration how a simple piece of R code is made 12,000 times faster: http://rwiki.sciviews.org/doku.php?id=tips:programming:code_optim2
? ? ?2. ?After you feel you have done the best you can with R, you might try coding the most compute intensive portion of the algorithm in a compiled language like C, C++ or Fortran. ?Then rerun Rprof, etc. ? ? ?3. ?After trying (or not) compiled code, it may be appropriate to consider "CRAN Task View: High-Performance and Parallel Computing with R". ?(From a CRAN mirror, select "Task Views" -> "HighPerformanceComputing: ?High-Performance and Parallel Computing with R".) ?You may also want to try the "foreach" package from Revolution Computing (revolutionanalytics.com). ?These capabilities can help you get the most out of a multi-core computer. ?NOTE: ?While your code is running, you can check the "Performance" tab in Windows Task Manager to see what percent of your CPUs and physical memory you are using. ?I mention this, because without "foreach" you might get at most 1 of your 4 CPUs running R. ?With "foreach", you might be able to get all of them working for you. ?Then after you have done this and satisfied yourself that you've done the best you can with all of this, I suggest you try the Amazon Cloud. ? ? ?If you have not already solved your problem with this and have not yet tried these three steps, I suggest you try this. ?It may take more of your time, but you will likely learn much that will help you in the future as well as help you make a better choice of a new computer if you ultimately decide to do that. ? ? ?Hope this helps. ? ? ?Spencer On 5/15/2011 8:28 AM, Gabor Grothendieck wrote:
On Fri, May 13, 2011 at 6:38 AM, Michael Haenlein <haenlein at escpeurope.eu> ?wrote:
I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
If you are on Windows press the Windows key and type in Power Options. ?When the associated dialog pops up choose Power Saver. ?Now your PC will use less power so it won't heat up so much although your performance could suffer a bit. Also ensure that there is sufficient air circulation around the machine.
To move this hardware-specific discussion off the R-help list, I strongly recommend the 'Thinkpad.com Support Community' (open community/non-Lenovo) with lots of experts and users: http://forum.thinkpads.com/ I've seen discussions on overheating/emergency shutdowns there. /Henrik
-- Spencer Graves, PE, PhD President and Chief Operating Officer Structure Inspection and Monitoring, Inc. 751 Emerson Ct. San Jos?, CA 95126 ph: ?408-655-4567
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
On 15/05/2011 3:02 PM, Aram Fingal wrote:
On May 13, 2011, at 6:38 AM, Michael Haenlein wrote:
Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
You didn't mention whether you are using a 64-bit OS or not. A single 32-bit process can not use more than 2 GB RAM. If your calculations would benefit from the full 8 GB RAM on your machine, you need to be able to run 64-bit R. My understanding is that, on Windows, you either have to install the OS as 32-bit and use all 32-bit software or install 64-bit Windows and run all 64-bit software. A Mac can run 32-bit and 64-bit software simultaneously and I'm not sure about Linux. In the case of Linux, it probably doesn't matter so much because most Linux software is available as open source and you can compile it yourself either way.
No, 64 bit Windows can run either 32 or 64 bit Windows programs.
I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance.
You need to evaluate whether RAM or raw processor speed is most critical for what you're doing. In my case, I upgraded my Mac Pro to 16 GB RAM and was able to do hierarchical clustering heatmaps overnight which previously took more than a week to compute. Using the Activity Monitor utility, it looks like some of the, even larger, heatmap computations would benefit from 32 GB RAM or more. Linux runs on the widest range of hardware and that allows you the greatest ability to shop around. If RAM is the deciding factor, then you can look around for a machine which can hold as much RAM as possible. If processor speed is the factor, then you can optimize for that. Windows runs on a reasonable array of hardware but has the disadvantage that the OS, itself, uses a lot of resources. The Mac has the advantage of flexibility. When you download the precompiled R package, it comes with both a 32-bit and a 64-bit executable. This is because 32-bit processes run a little faster if you don't need large amounts of RAM. If you do need the RAM, then you run the 64-bit version.
The same is true for Windows binaries on CRAN. Duncan Murdoch
On Sun, 15 May 2011, Duncan Murdoch wrote:
On 15/05/2011 3:02 PM, Aram Fingal wrote:
On May 13, 2011, at 6:38 AM, Michael Haenlein wrote:
Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating.
You didn't mention whether you are using a 64-bit OS or not. A single 32-bit process can not use more than 2 GB RAM.
And that is also false. For Windows, see the rw-FAQ. It is address space (not RAM) that is limited, and it is limited to 4GB *by definition* in a 32-bit process. Many OSes can give your process 4GB of address space, but may reserve some of it for the OS.
If your calculations would benefit from the full 8 GB RAM on your machine, you need to be able to run 64-bit R. My understanding is that, on Windows, you either have to install the OS as 32-bit and use all 32-bit software or install 64-bit Windows and run all 64-bit software. A Mac can run 32-bit and 64-bit software simultaneously and I'm not sure about Linux. In the case of Linux, it probably doesn't matter so much because most Linux software is available as open source and you can compile it yourself either way.
For the record, all modern 64-bit OSes on x86_64 cpus can do this provided you install 32-bit versions of core dynamic libraries. I run 32- and 64-bit R on 64-bit Linux, Solaris, FreeBSD, Darwin (the OS of Mac OS X), Windows .... As can AIX and IRIX on their CPUs.
No, 64 bit Windows can run either 32 or 64 bit Windows programs.
I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance.
You need to evaluate whether RAM or raw processor speed is most critical for what you're doing. In my case, I upgraded my Mac Pro to 16 GB RAM and was able to do hierarchical clustering heatmaps overnight which previously took more than a week to compute. Using the Activity Monitor utility, it looks like some of the, even larger, heatmap computations would benefit from 32 GB RAM or more. Linux runs on the widest range of hardware and that allows you the greatest ability to shop around. If RAM is the deciding factor, then you can look around for a machine which can hold as much RAM as possible. If processor speed is the factor, then you can optimize for that. Windows runs on a reasonable array of hardware but has the disadvantage that the OS, itself, uses a lot of resources.
Nothing like as much as Mac OS X, though. (I would say the main disadvantage of Windows for R is the slowness of the file systems.)
The Mac has the advantage of flexibility. When you download the precompiled R package, it comes with both a 32-bit and a 64-bit executable. This is because 32-bit processes run a little faster if you don't need large amounts of RAM. If you do need the RAM, then you run the 64-bit version.
The same is true for Windows binaries on CRAN.
And of e.g. the Fedora binaries.
Duncan Murdoch
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
Mr Fingal: please do! You are clearly unfamiliar with the R manuals.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
On Fri, May 13, 2011 at 5:25 PM, John C Frain <frainj at gmail.com> wrote:
Given the price of the Thinkpad X201 I would be very upset if it overheated and would contact my supplier for a replacement.
Consider getting an aluminium cooling pad, such as those provided by Zalman. It helps keeping my HP from overheating and shutting down. Liviu
I also understand that this CPU may slow down if it is overheated. ?You would probably get a better performance from a desktop workstation or a larger laptop with better ventilation. ?As already stated a large mumer of cpu's and multithreading are of little advantage when you are running standard R. John On 13 May 2011 11:38, Michael Haenlein <haenlein at escpeurope.eu> wrote:
Dear all, I'm currently running R on my laptop -- a Lenovo Thinkpad X201 (Intel Core i7 CPU, M620, 2.67 Ghz, 8 GB RAM). The problem is that some of my calculations run for several days sometimes even weeks (mainly simulations over a large parameter space). Depending on the external conditions, my laptop sometimes shuts down due to overheating. I'm now thinking about buying a more powerful desktop PC or laptop. Can anybody advise me on the best configuration to run R as fast as possible? I will use this PC exclusively for R so any other factors are of limited importance. Thanks, Michael Michael Haenlein Assocaite Professor of Marketing ESCP Europe Paris, France ? ? ? ?[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
-- John C Frain Economics Department Trinity College Dublin Dublin 2 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj at tcd.ie mailto:frainj at gmail.com
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Do you know how to read? http://www.alienetworks.com/srtest.cfm http://goodies.xfce.org/projects/applications/xfce4-dict#speed-reader Do you know how to write? http://garbl.home.comcast.net/~garbl/stylemanual/e.htm#e-mail