In this post: http://www.mail-archive.com/r-devel at r-project.org/msg13248.html Henrik Bengtsson observes that the R function serialize() can run very slowly when the argument connection is null and provides a function serialize2() that runs much faster. The post is dated 7/24/2008. Some code I wrote using package Rmpi was running slowly and I found out the bottleneck was exactly the function serialize(), which is called by some functions in Rmpi. So I inserted the command serialize <- serialize2 in my code and there was a dramatic acceleration of the relevant portions of the program. I have run the same code on two other computers and it did not run slowly. To be more specific: In all three computers I am running R 2.11.1. All three computers are made by HP. One dual core desktop PC (with no hyperthreading) running Windows XP 32-bit and one Intel i5 notebook (dual core with hyperthreading) running Windows 7 64-bit are not slowed down by serialize(). One dual core notebook (no hyperthreading) running Windows 7 64-bit executes serialize() very slowly. I used the function memory.limit() to increase memory allocation but that did not make any difference. Of course substituting serialize2 for serialize() is not an ideal situation, among other reasons because serialize2() needlessly writes to and reads from a connection (a temporary file). Does anyone have insights on what is going on and maybe better ways to deal with the issue? Thanks. Fernando Saldanha
serialize() slow
3 messages · Fernando Saldanha, Paul Johnson, Martin Morgan
On Sat, Aug 21, 2010 at 2:16 PM, Fernando Saldanha <fsaldan1 at gmail.com> wrote:
In this post: http://www.mail-archive.com/r-devel at r-project.org/msg13248.html Henrik Bengtsson observes that the R function serialize() can run very slowly when the argument connection is null and provides a function serialize2() that runs much faster. The post is dated 7/24/2008. Some code I wrote using package Rmpi was running slowly and I found out the bottleneck was exactly the function serialize(), which is called by some functions in Rmpi. So I inserted the command serialize <- serialize2 in my code and there was a dramatic acceleration of the relevant portions of the program.
If serialize2 fixes it for you, doesn't that answer your own question? You ask why the different computers may perform differently. Can I ask if there is any chance you have been fiddling around with alternative BLAS? I see a huge horrible Rmpi slowdown when GOTOBLAS2 is installed with threads allowed > 1. I had not heard of the serialize/serialize2 thing you mention, but I'm interested to try it to see what effect it has. I will let you know if I see any difference.
Paul E. Johnson Professor, Political Science 1541 Lilac Lane, Room 504 University of Kansas
On 08/21/2010 12:16 PM, Fernando Saldanha wrote:
In this post: http://www.mail-archive.com/r-devel at r-project.org/msg13248.html Henrik Bengtsson observes that the R function serialize() can run very slowly when the argument connection is null and provides a function serialize2() that runs much faster. The post is dated 7/24/2008.
This has likely been fixed in R-devel http://www.mail-archive.com/r-devel at r-project.org/msg20921.html Martin
Some code I wrote using package Rmpi was running slowly and I found out the bottleneck was exactly the function serialize(), which is called by some functions in Rmpi. So I inserted the command serialize <- serialize2 in my code and there was a dramatic acceleration of the relevant portions of the program. I have run the same code on two other computers and it did not run slowly. To be more specific: In all three computers I am running R 2.11.1. All three computers are made by HP. One dual core desktop PC (with no hyperthreading) running Windows XP 32-bit and one Intel i5 notebook (dual core with hyperthreading) running Windows 7 64-bit are not slowed down by serialize(). One dual core notebook (no hyperthreading) running Windows 7 64-bit executes serialize() very slowly. I used the function memory.limit() to increase memory allocation but that did not make any difference. Of course substituting serialize2 for serialize() is not an ideal situation, among other reasons because serialize2() needlessly writes to and reads from a connection (a temporary file). Does anyone have insights on what is going on and maybe better ways to deal with the issue? Thanks. Fernando Saldanha
_______________________________________________ R-sig-hpc mailing list R-sig-hpc at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
Martin Morgan Computational Biology / Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: Arnold Building M1 B861 Phone: (206) 667-2793