Parallel runs of an external executable with snow in local

Thanks Uwe but, actually, I did so.

Since “filetorun.exe” looks in the current folder for “input.txt”, I tried moving all needed files to a newly created temporary folder “tmp.id” (say, tmp.1) and running the executable. This works fine by doing it directly from the windows command line but not by doing it from R, since using:

#
System(“C:/Users/…/currentworkdirectory/temp.1/filetorun.exe”)
#

Causes “filetorun.exe” to look for “input.txt” in “C:/Users/…/currentworkdirectory”. So there’s no point on moving files to a folder, it seems that input file must be situated in the current R work directory. Does anybody know how to avoid this behaviour?

I hope I’ve explained that clearly,

Xavier Portell Canal, PhD candidate.
Department of Agri-food Engineering,
Universitat Polit?cnica de Catalunya

-----Uwe Ligges <ligges at statistik.tu-dortmund.de> ha escrit: -----

Per a: Xavier Portell/UPC <xavier.portell at upc.edu>
De: Uwe Ligges <ligges at statistik.tu-dortmund.de>
Data: 05/08/2012 07:46PM
a/c: r-help at r-project.org
Assumpte: Re: [R] Parallel runs of an external executable with snow in local



On 03.08.2012 19:21, Xavier Portell/UPC wrote:

Hi everyone,

I'm aiming to run an external executable (say filetorun.EXE) in parallel. The external executable collect needed data from a file, say "input.txt" and, in turn,generates several output files, say "output.txt". I need to generate "input.txt", run the executable and keep "input.txt" and "output.txt". I'm using Windows 7, R version 2.15.1 (2012-06-22) on RStudio and platform: i386.pc.mingw32/i386 (32-bit).

My first attempt was a R code which, by using
    System("filetorun.EXE", intern = F, ignore.stdout = F,
            ignore.stderr = F, wait = T, input = NULL,
            show.output.on.console = T, minimized = F, invisible = T))
, ran the executable and kept required files to a conveniently named folder. After that I changed my previous R script so I could use the function lapply().This script apparently worked fine.

Finally, I tried to parallelize the problem by using snow and parLapply(). The resulting script looks like this:

## Not run
#
library(snow)cl <- makeCluster(3, type = "SOCK")
clusterExport(cl,list('param.esp','copy.files','for12.template','program.executor'))
parLapply(cl,a.list,a.function))stopCluster(cl)
#
##End not run

Although it runs, the parallelized version is messing up the input parameters to pass to the executable (see table below, where parameters P1 and P2 are considered. ".s" comes from the serial code and ".p" from the parallelized one):
    s r P1.s P2.s P1.p P2.p
1 1 1  1.0 3.00  2.0 3.00
2 2 1  1.5 3.00  2.0 3.75
3 3 1  2.0 3.00  2.0 3.00
4 4 1  1.0 3.75  1.5 3.00
5 5 1  1.5 3.75  1.5 3.00
6 6 1  2.0 3.75  2.0 3.75

My first thought to avoid the described behaviour was creating a temporary file, say "tmp.id" with id being an identification run number, and copying "filetorun.EXE" and "Input.txt" to "tmp.id". However, while doing so, I realised that although running the correct "filetorun.EXE" copy (i.e., the one in "tmp.id") R looks for "input.txt" in the work directory.

Not sure about the real setup, but you can actually specify the path,
not only filenames.

Uwe Ligges

I've been looking thoroughly for a solution but I got nothing.

Thanks for any help in advance,


Xavier Portell Canal

PhD candidate
Department of Agri-food engineering,
Universitat Polit?cnica de Catalunya