parallel: Race-condition concern regarding graphics devices in a multi-thread environment

On Fri, Apr 5, 2013 at 2:44 PM, Simon Urbanek
Henrik,

there are no threads.
Oh my... all this time I've thought mc* were working with threads.
...despite the 'parallel' vignette writing it in black and white: "In
principle the workers could be implemented by threads\footnote{only
`in principle' since the R interpreter is not thread-safe.} or
lightweight processes, but in the current implementation they are full
processes."  Note to self: lapply(rep(c(9,14), times=100),
fortunes::fortune)
Hence you're on the wrong track there, it has nothing to do with device numbers/devices.The device number will be the same in all processes and that's ok.
Got it; at the moment the main process is forked, each child *process*
gets a copy of the existing list of devices (devices which they then
may interact with but that's bad style - or is ignored?!?), whereas
any newly created graphics device is unique to each child process.
Device indices are local to each child processes.   In case someone
else follows this or finds this thread later, the following example
illustrates this:

library("parallel");
jpeg("foo0.jpg");
idx <- dev.cur();
plot(1:10, main="main");

dummy <- function(i) {
  before <- dev.list();
  filename <- sprintf("foo%d.png", i);
  png(filename);
  plot(1:10, main=i);
  # No dev.off(); tests what happens if you forget
  # to close device (see below).
  list(before=before, after=dev.list());
}

res <- mclapply(1:2, FUN=dummy, mc.cores=2);
str(res);
## List of 2
##  $ :List of 2
##   ..$ before: Named int 2
##   .. ..- attr(*, "names")= chr "jpeg"
##   ..$ after : Named int [1:2] 2 3
##   .. ..- attr(*, "names")= chr [1:2] "jpeg" "png"
##  $ :List of 2
##   ..$ before: Named int 2
##   .. ..- attr(*, "names")= chr "jpeg"
##   ..$ after : Named int [1:2] 2 3
##   .. ..- attr(*, "names")= chr [1:2] "jpeg" "png"

# When child processes finishes, any open devices are closed.
str(dev.list());
##  Named int 2
## - attr(*, "names")= chr "jpeg"

# Close the opened jpeg device.
dev.off(idx);

# All devices closed
str(dev.list())
##  NULL
The issue here really is which png() pack-end are you using? (You didn't say).
I was considering the general case (where I don't know), e.g.
dev.new(<user specified>).
Some back-ends (like X11) cannot be run in forked environment, so you can't use them.  I don't use png() myself, but I do know that CairoPNG() from the Cairo package works when forked.
Test/example:

dummy <- function(i) {
  x11();
  plot(1:10, main=i);
}

res <- mclapply(1:2, FUN=dummy, mc.cores=2);
## Warning message:
## In mclapply(1:2, FUN = dummy, mc.cores = 2) :
##   all scheduled cores encountered errors in user code
print(res);
## [[1]]
## [1] "Error in x11() : a forked child should not open a graphics device\n"
## [[2]]
## [1] "Error in x11() : a forked child should not open a graphics device\n"

Thanks for your help Simon,

Henrik
Cheers,
Simon

On Apr 5, 2013, at 12:23 PM, Henrik Bengtsson <hb at biostat.ucsf.edu> wrote:

Hi,

I'm trying to figure out how to safely make sure that I close the same
graphics device that I opened earlier in a thread (and not one opened
by a parallel thread).  In a *single-thread* environment, one can do
the following to open and close a device:

makePlot <- function(i) {
 filename <- sprintf("foo%d.png", i);
 png(filename);
 idx <- dev.cur();
 on.exit(dev.off(idx));
 plot(1:10, col=i);
 title(main=i);
 filename;
} # makePlot()

However, in a *multi-thread* environment, e.g.

res <- mclapply(1:4, FUN=makePlot, mc.cores=4);

the following race-condition problem may occur, because of the
non-atomicity(?) of (i) png() and (ii) dev.cur():

1. Thread #1: png("foo1.png")
2. Thread #2: png("foo2.png")
3. Thread #1: idx <- dev.cur();  # idx == 3
4. Thread #2: idx <- dev.cur();  # idx == 3 (same device!)
...
5. Thread #1: dev.off(idx); # Closes device #3
6. Thread #2: plot(1:10, col=2); # Trying to plot, which opens a new
on-screen device (or to another active device) since device #3 is
closed.
7. Thread #2: dev.off(idx) # Trying to close device #3, which is already closed.

On this topic, there are some clues/notes on concern in the vignette
of the 'parallel' package, but not enough to answer my concerns/solve
my problems.  Any other references/discussion on this (other that
source code)?

Q1. Is there already a built-in protection against the above race condition?

Q2. If not on Q1, is there a way/strategy to get hold of the device
(index) for the graphics devices that was opened by png() without
using dev.cur()?

If not on Q2, what about transitioning to have graphics device
functions return the opened device (index) (cf. connections), e.g. idx
<- png(...).  A quick look at ?png and ?x11 show that those functions
does not return anything (although code inspections show that they may
return something, but always NULL when I try).

Comments?

Henrik

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

parallel: Race-condition concern regarding graphics devices in a multi-thread environment

Thread (3 messages)