R-devel Digest, Vol 90, Issue 20
On Aug 22, 2010, at 3:32 AM, Laurent wrote:
On 21/08/10 23:31, Simon Urbanek wrote:
On Aug 21, 2010, at 8:46 AM, Laurent wrote:
On 21/08/10 12:00, r-devel-request at r-project.org wrote:
On Aug 20, 2010, at 1:59 PM, Matt Shotwell wrote:
On Fri, 2010-08-20 at 12:58 -0400, Sharpie wrote:
Donald Paul Winston wrote:
(...)
Donald Paul Winston wrote:
It appears R insists on directing plot output to a file. Is their a graphics device that keeps the output in memory so it can be returned to my java app as a stream of bytes? If I have to wait for it to write to a "unique file" and then read it back again then I don't think that's going to work. My web app needs to support hundreds of concurrent clients.
As far as I know all R graphics output that does not go to a screen device, such as an X window, must be directed to some sort of file. I am not aware of a graphics device that provides output to something other than a screen or a file, but there very well may be such a device in existence.
For experimentation purpose, the rpy2 project is finalizing a system to allow Python-written graphical devices (no C-level R extensions, just Python). Beside other niceties, it will allow working around the lack of connection API in R for graphics. Since Python makes the use of file-like objects straightforward, we plan providing devices that are streams (nice for serving graphics from web applications without having to worry about temp files).
Well, that doesn't help with the issue we raised since you still cannot use R connections.
I do think it will in the use-case proposed: have R serve graphics without having to worry about writing temp files.
Actually I have completely forgotten that I have implemented that in Cairo already -- you can fetch the current image from any image Cairo device (without even closing it) using the .image() function. So for example you could use something like: Cairo(640, 480, "/dev/null", "png") ## the file is actually not used in this case at all so you can use anything plot(...) ## get the image (as a reference) i=Cairo:::.image(dev.cur()) ## if your code can take direct pointer (RGBA), you're done. If you want to further convert it to RAW and more, then: r=Cairo:::.ptr.to.raw(i$ref,0,i$width*i$height*4) dim(r) = c(4, i$width, i$height) p=writePNG(r, raw()) ## p is now the raw PNG content - ready to be served to the client as image/png -- no files involved
Python is somewhat used in the web applications world, and what is proposed allows serving graphics without a temp file in sight.
Somewhat is the key word ;) but nonetheless I'm not sure I understand how you plan to do that because you'd have to embed R for this which will cause you to have to initialize R which is simply too slow for any serious web processing .. If you go the other way round (like starting with Rserve) then you don't have Python as a starting point so you'd have to load Python for that in the first place which seems like an overkill. Cheers, Simon
It's trivial to modify any of the existing devices (e.g. Cairo as I said) to support in-memory storage as they already support that internally - certainly much easier that to mess with Python etc.
I have been neck-deep in R C-level code for devices when working on that, and ended up with the exact opposite opinion.
Nonetheless, yes, it is another way along the lines of xGD, JavaGD etc.
Thanks for the pointer. I did not know about them. JavaGD might well be the most helpful thing then (if Donald had in mind to do the web handling made in Java). Best, Laurent
Cheers, Simon
This was essentially the conclusion of Donald's earlier thread...
The functionality you could describe could be implemented by writing a R graphics device that forwards the R plotting commands to... well anywhere you want, really. As the author of an R graphics device, I can say the trickiest part of such an undertaking will be calculating font metrics so that R can properly position text in the graphics. Everything else is very straight-forward.
I believe Donald wants the_rendered_ output. That is, a stream of bytes that represent a png, jpeg, etc. Toward this end, we have already discussed using an OS-level fifo to avoid disk I/O, and I think this is the easiest route for Donald. Alternatively, Donald might look into the X11 graphics driver, where Cairo is used to write pngs. In particular, the in_do_saveplot function (src/main/modules/X11/devX11.c) calls cairo_surface_write_to_png, in order to save a png to file. Cairo also provides the cairo_surface_write_to_png_stream function, which might be used to send png data to a memory buffer (i.e. a raw vector). I don't think it would be too difficult to modify in_do_saveplot to accommodate this.
That is all nice, it will require you to hack R (well, there is the Cairo device so you could make the change outside of R instead). As always, the reason is buried deep inside -- the lack of connection API. If we had that, devices would take a connection instead of the file name and all would be well since you could use in-memory connections instead of files ...;)
May I dare asking about what happened to past offers to alleviate the problem ? There is at least one patch (contributed 4 years ago) http://rwiki.sciviews.org/doku.php?id=developers:r_connections_api that remained seemingly ignored, and subsequent requests for updates (or patch submission policies) remained similarly unanswered. A recent thread showed unexpected progress, with the eventual possibility of accepting a patch being worded. http://www.mail-archive.com/r-devel at r-project.org/msg20276.html Unfortunately the thread drifted into legalese invocations, and despite the author of the patch complying to all demands regarding the licensing flavour for his contribution the patch seems to have joined (all ?) other submitted patches. (I don't get anything when running: svn log -r {2010-04-27}:HEAD | grep -i Shotwell ). Are there such patches included in the Revolution R sources, or are there plans to do so ? Laurent
Cheers, Simon