mclapply THE_PROCESS_HAS_FORKED error

Dear all,

I have got a question regarding the multicore package, used on a macbook
pro with 4 cores. I have code that can be split into 2 independent pieces,
such that I call

result <- mclapply(1:2, function(x) myfun(x) )

It works perfectly - most of the time. Sometimes I get this error message:

Break on
__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()
to debug.

This happens particularly after I called mclapply over and over again in
the same session. I have a suspicion that some forked processes are not
properly terminated after the function exits (I can see them in the
activity monitor). Notice I do not change option mc.cleanup (i.e. it's at
default TRUE).

I imagine a source of error is myfun (too big to paste here) - the only
thing I can say is that it runs without problem in lapply.

My biggest concern is that I want to run this function on a unix cluster.
I'm not sure if what I'm saying makes a lot of sense but I want to make
sure that I don't accumulate many idle R processes and run into that
problem.

Hi Steve,

never tried the --vanilla approach, I guess i should. There are no plots in
the code, but thanks for pointing that out, I'll watch out for this in
future.

In some cases the function actually returned the correct value but R.app
was spitting out the error message over and over again. in the activity
monitor the 2 forked processes got stuck at 100% CPU usage. in other cases
it crashed.

I'm sorry if the info I am giving is very generic. Any experience you can
share is useful - thanks.

Florian


On 27 June 2013 19:34, Steve Lianoglou <lianoglou.steve at gene.com> wrote:

Hi,

On Thu, Jun 27, 2013 at 11:09 AM, Florian Oswald
<florian.oswald at gmail.com> wrote:

Dear all,

I have got a question regarding the multicore package, used on a macbook
pro with 4 cores. I have code that can be split into 2 independent

pieces,

such that I call

result <- mclapply(1:2, function(x) myfun(x) )

It works perfectly - most of the time. Sometimes I get this error

message:

Break on

__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()

to debug.

This happens particularly after I called mclapply over and over again in
the same session. I have a suspicion that some forked processes are not
properly terminated after the function exits (I can see them in the
activity monitor). Notice I do not change option mc.cleanup (i.e. it's at
default TRUE).

I imagine a source of error is myfun (too big to paste here) - the only
thing I can say is that it runs without problem in lapply.

My biggest concern is that I want to run this function on a unix cluster.
I'm not sure if what I'm saying makes a lot of sense but I want to make
sure that I don't accumulate many idle R processes and run into that
problem.

Does this problem only happen when you are running R through R.app? I
mean, if you fire up R from the terminal with `R --vanilla`, then try
your stuff again, do you see it happen?

Other things to look out for: is there plotting code in `myfun`?

-steve

--
Steve Lianoglou
Computational Biologist
Bioinformatics and Computational Biology
Genentech

	[[alternative HTML version deleted]]