concurrent requests (Rook, but I think the question is more general)

On 24/10/12 8:53 PM, Simon Urbanek wrote:

On Oct 24, 2012, at 2:13 PM, Richard D. Morey wrote:

This question involves Rook, but I think the answer will be general enough that it pays to post here. At any rate, I don't know enough to know whether this is a Rook only issue or a general R issue.

Here's what I'd like to do (and indeed, have code that should do this):

1. Start R, Rook
2. Start an analysis via a HTTP request to Rook. This analysis uses .Call() to some compiled C code, if that matters. The C code calls a callback function to update a variable with its progress.
3. While the analysis is happening, use Rook to obtain current status with an HTTP request

You can't. R doesn't support threading so it's simply not possible to have an asynchronous eval. The R HTTP server works by simply enqueuing an eval to run while R is idle, it can't do that if R is busy. (Note that the HTTP server was *only* designed for the internal help).

What you can do is have your C code start another thread that reports the progress when asked e.g. on a socket, but that thread is not allowed to call any R API so you want that progress to be entirely in your C code.

How can I start a new thread? By running R again from the command line, or is there a better way?

Note that if your C code is robust enough, it can call R_CheckUserInterrupt() to allow external events to happen, but a) your C code must in that case be prepared for early termination (clean up memory etc.) and b) I don't remember if the httpd is allowed to run during interrupt check on all platforms - you may want to check that first.

I do use R_CheckUserInterrupt(), but if I understand what you're saying then given that it doesn't work, httpd must not run during the interrupt check. At least, on OSX, which is what I'm testing on.

Cheers,
Simon

The problem is that once the analysis starts, Rook does not respond to requests. All of the status requests to Rook pile up, and then are answered when the analysis (step 2) is done. Here is some example code to demonstrate what the issue:

##########

library(Rook)
s <- Rhttpd$new()
s$add(
 name="pingpong",
 app=Rook::URLMap$new(
   '/ping' = function(env){
     req <- Rook::Request$new(env)
     res <- Rook::Response$new()
     res$write('This is ping.')
     Sys.sleep(20)
     res$finish()
   },
   '/pong' = function(env){
     req <- Rook::Request$new(env)
     res <- Rook::Response$new()
     res$write("This is pong.")
     res$finish()
   },
   '/?' = function(env){
     req <- Rook::Request$new(env)
     res <- Rook::Response$new()
     res$redirect(req$to_url('/pong'))
     res$finish()
   }
 )
)

s$start(quiet=TRUE)
s$browse('pingpong')

#############################

If you request /ping, R runs Sys.sleep() for 20 seconds. This is where my .Call() statement would be. While the .Call() (Sys.sleep()) function is doing its thing, I need to get Rook to respond on /pong (which would simply respond with the progress), but if you run this code, request /ping, then immediately request /pong, you'll see that the /pong request will not be answered until the Sys.sleep() is done.

Of course, for a progress report to be useful, the requests have to be answered immediately. Is this a Rook issue, or an R issue? Or am I asking something unreasonable?