Skip to content

rzmq package

3 messages · Whit Armstrong, Daniel Cegiełka, Dirk Eddelbuettel

Original
Whit Armstrong
# 
Just a quick post on a new package I've been wrapping up, rzmq.

https://github.com/armstrtw/rzmq

Finding inspiration in JD Long's segue package, and frustration with
the config steps involved in dynamically updating the older debian
distribution that Amazon uses for it's emr machines, I decided to try
to hit the ec2 machines directly using my own ami.

The zmq messaging patterns allow one to distribute jobs across many
nodes, but for now a simple example with only 1 micro instance.

the remote server:

ubuntu at ip-10-243-90-36:~$ cat remote.server2.r
#!/usr/bin/env Rscript

library(rzmq)
context = init.context()
in.socket = init.socket(context,"ZMQ_PULL")
bind.socket(in.socket,"tcp://*:5557")

out.socket = init.socket(context,"ZMQ_PUSH")
bind.socket(out.socket,"tcp://*:5558")

while(1) {
    msg = receive.socket(in.socket);
    fun <- msg$fun
    args <- msg$args
    print(args)
    ans <- do.call(fun,args)
    send.socket(out.socket,ans);
}
ubuntu at ip-10-243-90-36:~$


and the locally executed code:

estimatePi <- function(seed) {
    set.seed(seed)
    numDraws <- 1e5

    r <- .5 #radius... in case the unit circle is too boring
    x <- runif(numDraws, min=-r, max=r)
    y <- runif(numDraws, min=-r, max=r)
    inCircle <- ifelse( (x^2 + y^2)^.5 < r , 1, 0)

    sum(inCircle) / length(inCircle) * 4
}


print(system.time(ans <- zmq.lapply(as.list(1:1e2),
                                    estimatePi,

execution.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5557",

sink.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5558")))

print(mean(unlist(ans)))

yields:
warmstrong at krypton:~/dvl/zmq.test/test.ec2$ Rscript lapply.exmaple.r
   user  system elapsed
  0.010   0.010   7.007
[1] 3.140964


Anyway, I'll post up a better example tomorrow that actually uses more
than one machine.

-Whit
Dirk Eddelbuettel
#
On 28 September 2011 at 22:54, Whit Armstrong wrote:
| Just a quick post on a new package I've been wrapping up, rzmq.
| 
| https://github.com/armstrtw/rzmq

Wicked!  There are so many of us here who are ZeroMQ fanboys.  Were you aware
that Bryan Lewis is working on one too?
 
| Finding inspiration in JD Long's segue package, and frustration with
| the config steps involved in dynamically updating the older debian
| distribution that Amazon uses for it's emr machines, I decided to try
| to hit the ec2 machines directly using my own ami.
| 
| The zmq messaging patterns allow one to distribute jobs across many
| nodes, but for now a simple example with only 1 micro instance.
| 
| the remote server:
| 
| ubuntu at ip-10-243-90-36:~$ cat remote.server2.r
| #!/usr/bin/env Rscript
| 
| library(rzmq)
| context = init.context()
| in.socket = init.socket(context,"ZMQ_PULL")
| bind.socket(in.socket,"tcp://*:5557")
| 
| out.socket = init.socket(context,"ZMQ_PUSH")
| bind.socket(out.socket,"tcp://*:5558")
| 
| while(1) {
|     msg = receive.socket(in.socket);
|     fun <- msg$fun
|     args <- msg$args
|     print(args)
|     ans <- do.call(fun,args)
|     send.socket(out.socket,ans);
| }
| ubuntu at ip-10-243-90-36:~$
| 
| 
| and the locally executed code:
| 
| estimatePi <- function(seed) {
|     set.seed(seed)
|     numDraws <- 1e5
| 
|     r <- .5 #radius... in case the unit circle is too boring
|     x <- runif(numDraws, min=-r, max=r)
|     y <- runif(numDraws, min=-r, max=r)
|     inCircle <- ifelse( (x^2 + y^2)^.5 < r , 1, 0)
| 
|     sum(inCircle) / length(inCircle) * 4
| }
| 
| 
| print(system.time(ans <- zmq.lapply(as.list(1:1e2),
|                                     estimatePi,
| 
| execution.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5557",
| 
| sink.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5558")))
| 
| print(mean(unlist(ans)))
| 
| yields:
| warmstrong at krypton:~/dvl/zmq.test/test.ec2$ Rscript lapply.exmaple.r
|    user  system elapsed
|   0.010   0.010   7.007
| [1] 3.140964
| 
| 
| Anyway, I'll post up a better example tomorrow that actually uses more
| than one machine.

You have a bug in that you use Rscript in lieu of littler :)

Nicely done. Looking forward to more.

Cheers, Dirk