multithreading calling from the rpy Python package

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

There are several sets of notes on threading off
http://developer.r-project.org page--somewhat old but still relevant.
The Python approach is discussed there.  That approach, which gives
concurrency but not parallelism, is in principle fleasible for R but
getting from here to there is non-trivial given that we have some
unique issues related to FORTRAN semantics as well as how many R
packages are written.  It may happen yet, but probably later rather
than sooner.

To follow up on Luke's comments..

We can partially automate the work to get multiple evaluators in R.
And we are getting very close to having the underlying tools to do this.
But it still remains to be seen whether the extra work to introduce
threads is warranted. Will people actually use them in R and will it
have a significant impact on the computations or simply make writing
GUIs within R slightly easier to manage?
The design of R may not be ideal for high performance computing and
a new architecture and system explicitly for more specialized
computation in the short term may be warranted. Some of us are thinking
about this from a variety of different perspectives.  Lee Edlefsen h
has some intersting work at exametrix.com


One of the reasons I am hesitant to use Python as a framework on
which to build a new system is the "thread-safe but not threadable"
issue. Also, it is not easily extensible in an object oriented manner
and this is a big issue for evolvability and user extensions to a system.

Best,

luke

On Fri, 20 Oct 2006, Ren? J.V. Bertin wrote:

Since Python has been mentioned in this context: Could not Python's
threading model and implementation serve as a guideline?

From a few simple benchmarks I've run, it seems as if the Python

interpreter itself is thread-safe but not threadable. That is, when I
run something "pure Python" like a recursive function that returns the
nth Fibonacci number in parallel, there is no speed-up for 2 threads
on a dual-processor machine. However, calling sleep in parallel does
scale down with the number of threads, even on a single-processor ;)

Real-life code does tend to speed up somewhat, though never as much as
one would hope.

Just an idea...

Ren?

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

- --
Duncan Temple Lang                    duncan at wald.ucdavis.edu
Department of Statistics              work:  (530) 752-4782
4210 Mathematical Sciences Building   fax:   (530) 752-7099
One Shields Ave.
University of California at Davis
Davis,
CA 95616,
USA
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFOQBY9p/Jzwa2QP4RAoD9AJwOxnuLKp+1pkregjsmgi0XQczqDgCbBFsB
eR4bQXKHNUBk6RILk0kxVg4=
=1YKW
-----END PGP SIGNATURE-----