Skip to content

[Bioc-devel] Problem running Perl script with R-devel

16 messages · Dan Tenenbaum, Henrik Bengtsson, Warden, Charles

#
Hi Charles,


----- Original Message -----
Can you provide the exact commands you entered which caused the error? As well as the output of sessionInfo()?
Thanks,
Dan
#
Hi Dan,

Thanks for your prompt response. To simplify things, I'll provide the code for the truncated dataset:

library("COHCAP")

dir <- system.file("extdata", package="COHCAP")
beta.file <- file.path(dir,"GSE42308_truncated.txt")
sample.file <- file.path(dir,"sample_GSE42308.txt")
project.folder <- "T:\\BIC\\cwarden\\Methylation_Package\\Protocol_Exchange\\minfi_example"
project.name <- "450k_avg_by_island_test"
beta.table <- COHCAP.annotate(beta.file, project.name, project.folder,
				platform="450k-UCSC")


filtered.sites <- COHCAP.site(sample.file, beta.table, project.name,
				project.folder, ref="parental")

And here is the sessionInfo() and warning() outputs:

Warning message:
running command 'perl C:/Program Files/R/R-devel/library/COHCAP/Perl/create_wig_files.pl T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig/temp.txt T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig' had status 2
R Under development (unstable) (2014-01-20 r64849)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] COHCAP_0.99.5     COHCAPanno_0.99.2 WriteXLS_3.3.1   

loaded via a namespace (and not attached):
[1] tools_3.1.0

Please let me know if you need anything else.

Thanks,
Charles

-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 10:51 AM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel

Hi Charles,


----- Original Message -----
Can you provide the exact commands you entered which caused the error? As well as the output of sessionInfo()?
Thanks,
Dan
#
Hi Charles,

----- Original Message -----
I can't reproduce the problem on either mac or windows. I changed project.folder to tempdir() but otherwise left your code alone.

Do you have a perl command in your PATH? 

What's the output of traceback() after you get the error? What are the arguments being passed to system()?

Dan
#
Hi Dan,

Here is the trackback output:
2: read.table(sample.file, header = F, sep = "\t")
1: COHCAP.site(sample.file, beta.table, project.name, project.folder, 
       ref = "parental")
I have no problem running Perl via command line (and it works with the non-devel version of R).

This is the command being run within COHCAP (with the specific values filled in):

cmd <- 'perl C:/Program Files/R/R-devel/library/COHCAP/Perl/create_wig_files.pl T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig/temp.txt T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig'

system(cmd, intern=TRUE, wait=TRUE)

Please let me know if you need anything else.

Best,
Charles

-----Original Message-----
From: Warden, Charles 
Sent: Monday, January 27, 2014 10:57 AM
To: 'Dan Tenenbaum'
Cc: bioc-devel at r-project.org
Subject: RE: [Bioc-devel] Problem running Perl script with R-devel

Hi Dan,

Thanks for your prompt response. To simplify things, I'll provide the code for the truncated dataset:

library("COHCAP")

dir <- system.file("extdata", package="COHCAP")
beta.file <- file.path(dir,"GSE42308_truncated.txt")
sample.file <- file.path(dir,"sample_GSE42308.txt")
project.folder <- "T:\\BIC\\cwarden\\Methylation_Package\\Protocol_Exchange\\minfi_example"
project.name <- "450k_avg_by_island_test"
beta.table <- COHCAP.annotate(beta.file, project.name, project.folder,
				platform="450k-UCSC")


filtered.sites <- COHCAP.site(sample.file, beta.table, project.name,
				project.folder, ref="parental")

And here is the sessionInfo() and warning() outputs:

Warning message:
running command 'perl C:/Program Files/R/R-devel/library/COHCAP/Perl/create_wig_files.pl T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig/temp.txt T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/450k_avg_by_island_test_wig' had status 2
R Under development (unstable) (2014-01-20 r64849)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] COHCAP_0.99.5     COHCAPanno_0.99.2 WriteXLS_3.3.1   

loaded via a namespace (and not attached):
[1] tools_3.1.0

Please let me know if you need anything else.

Thanks,
Charles

-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 10:51 AM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel

Hi Charles,


----- Original Message -----
Can you provide the exact commands you entered which caused the error? As well as the output of sessionInfo()?
Thanks,
Dan
#
----- Original Message -----
Does the directory 
T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/4
exist prior to running the code? I tried running your code snippet on a directory that didn't exist and got an error (not the same error as you). 

It could be some other issue related to the output directory (perhaps you don't have permission to write to it).

What if you set project.folder to tempdir() and run the code again, does it work?

Dan
#
----- Original Message -----
Note also that this path has a mixture of forward and back slashes. What happens if you change it to use all one or the other? 

Dan
#
Hi Dan,

1) When I first ran the code, the folder didn't exist (it is created from the COHCAP function).  When I run the same set of code in my non-devel version of R (with COHCAP installed via R CMD INSTALL, instead of biocLite), all the files are created properly (in the exact same location)

2)If I set project.folder to tempdir(), then I encounter the same problem.

3) Yes, I wish that the slashes and backslashes were consistent.  I am not actually specifying them directly. Instead, I am using system.path to add the subfolders within project.folder.  In other words, this is the actual code (I was just trying to show you what the text of the command looks like in this specific case):

	temp.stat.file <- file.path(wig.folder, "temp.txt")
	Perl.Path <- file.path(path.package("COHCAP"), "Perl")
	perl.script <- file.path(Perl.Path , "create_wig_files.pl")
	write.table(stat.table, temp.stat.file, quote=F, row.names=F, sep="\t")
	cmd <- paste("perl",perl.script, temp.stat.file, wig.folder, sep=" ")
	system(cmd, intern=TRUE, wait=TRUE)

I hope this clarifies things.  Please let me know if you still have any questions.

Thanks,
Charles

-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 11:27 AM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel



----- Original Message -----
Does the directory 
T:\BIC\cwarden\Methylation_Package\Protocol_Exchange\minfi_example/CpG_Site/4
exist prior to running the code? I tried running your code snippet on a directory that didn't exist and got an error (not the same error as you). 

It could be some other issue related to the output directory (perhaps you don't have permission to write to it).

What if you set project.folder to tempdir() and run the code again, does it work?

Dan
#
----- Original Message -----
What do you see if you run 
Sys.getenv("PATH")

?

Does it include the directory that contains perl.exe?

If not, then perl is not in your PATH. Note that running
echo %PATH% 
in the command shell and running Sys.getenv("PATH") in R can give different results because R modifies the path.

Dan
#
Hi Dan,

Here is the output from the requested command:

C:\\Perl64\\site\\bin;C:\\Perl64\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\MATLAB\\R2011b\\runtime\\win64;C:\\Program Files\\MATLAB\\R2011b\\bin;C:\\Program Files\\MATLAB\\R2010b\\runtime\\win64;C:\\Program Files\\MATLAB\\R2010b\\bin;C:\\Program Files (x86)\\Common Files\\Roxio Shared\\DLLShared\\;C:\\Program Files (x86)\\QuickTime\\QTSystem\\;c:\\Program Files (x86)\\Microsoft SQL Server\\100\\Tools\\Binn\\VSShell\\Common7\\IDE\\;c:\\Program Files (x86)\\Microsoft SQL Server\\100\\Tools\\Binn\\;c:\\Program Files\\Microsoft SQL Server\\100\\Tools\\Binn\\;c:\\Program Files (x86)\\Microsoft SQL Server\\100\\DTS\\Binn\\;C:\\Program Files\\Java\\jdk1.7.0_05\\bin;C:\\Program Files (x86)\\Subversion\\bin;C:\\Rtools\\bin;C:\\Rtools\\gcc-4.6.3\\bin;C:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\;C:\\Program Files\\R\\R-3.0.2\\bin\\x64\\;C:\\Program Files\\gs\\gs9.10\\bin"

"C:\\Perl64\\site\\bin" contains perl.exe and perl5.12.4.exe.  Also, this command works with the following output:
This is perl 5, version 12, subversion 4 (v5.12.4) built for MSWin32-x64-multi-thread
(with 9 registered patches, see perl -V for more detail)

Copyright 1987-2010, Larry Wall

Binary build 1205 [294981] provided by ActiveState http://www.ActiveState.com
Built Jun 20 2011 18:11:54

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

So, I assume the problem occurs after the perl command is run (and, for whatever reason, the problem only occurs with R-devel).

Thanks,
Charles
-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 11:54 AM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel



----- Original Message -----
What do you see if you run 
Sys.getenv("PATH")

?

Does it include the directory that contains perl.exe?

If not, then perl is not in your PATH. Note that running
echo %PATH% 
in the command shell and running Sys.getenv("PATH") in R can give different results because R modifies the path.

Dan
#
Some quick ideas:

1. Before making the full call, call res <- system2(perlbin, "-v") to
assert no errors about running Perl itself.  If an error, catch it and
given an information message ("Failed to run 'perl -v'" or something).
That will be user friendly as well for future users who run into
troubles.

2. Before calling system(), you can also assert that the script and
the input files are accessible, cf. file_test("-f") and file.access().
 If not, give an informative error message.

3. Do res <- system(cmd, intern=TRUE, wait=TRUE) and report on 'res'
when there's an error.  It will show the output of your Perl session.

4. You can use x <- normalizePath(x) to create OS-consistent paths/pathnames.

My $.02

/Henrik
On Mon, Jan 27, 2014 at 10:56 AM, Warden, Charles <cwarden at coh.org> wrote:
#
----- Original Message -----
What happens if you just run the command above in isolation? 

Is T: a networked drive?
what does tempdir() display? Is it also a directory on drive T:? 
If so, try using a directory on a physical (local) drive as your output directory.

I'm not sure if the error code 2 that you are seeing is from perl or from R, but the googles (http://www.perlmonks.org/?node_id=274896) suggests that error 2 is "no such file or directory" so it could be a problem with one of the parameters you're passing to perl. 

BTW, I tried the following (forgive the bad line wrapping, that's what happens when I paste from windows into my mac):
[1] "E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl"
[1] "perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C:\\Users\\BI
OCBU~1\\AppData\\Local\\Temp\\2\\Rtmp4KS893/temp.txt C:\\Users\\BIOCBU~1\\AppData\\Local\\
Temp\\2\\Rtmp4KS893"
Uncaught exception from user code:
        Could not open C:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt! at E:/b
iocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35.
 at E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35
character(0)
attr(,"status")
[1] 2
Warning message:
running command 'perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C
:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt C:\Users\BIOCBU~1\AppData\Local\
Temp\2\Rtmp4KS893' had status 2
So this is complaining because the temp.txt file does not exist.

Henrik's suggestions are good too.

Dan
#
Hi Henrik,

Thanks for your feedback.

No matter what, I will implement suggestion #3 in the next version of COHCAP.  I agree that is important.

Best,
Charles

-----Original Message-----
From: henrik.bengtsson at gmail.com [mailto:henrik.bengtsson at gmail.com] On Behalf Of Henrik Bengtsson
Sent: Monday, January 27, 2014 12:38 PM
To: Warden, Charles
Cc: Dan Tenenbaum; bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel

Some quick ideas:

1. Before making the full call, call res <- system2(perlbin, "-v") to
assert no errors about running Perl itself.  If an error, catch it and
given an information message ("Failed to run 'perl -v'" or something).
That will be user friendly as well for future users who run into
troubles.

2. Before calling system(), you can also assert that the script and
the input files are accessible, cf. file_test("-f") and file.access().
 If not, give an informative error message.

3. Do res <- system(cmd, intern=TRUE, wait=TRUE) and report on 'res'
when there's an error.  It will show the output of your Perl session.

4. You can use x <- normalizePath(x) to create OS-consistent paths/pathnames.

My $.02

/Henrik
On Mon, Jan 27, 2014 at 10:56 AM, Warden, Charles <cwarden at coh.org> wrote:
#
Hi Dan,

tempdir() is on the C: drive.

I will try outputting the system() result as a warning.  For some reason, I thought this should have already appeared in the warning() thread, and I was wondering why I didn't see any more specific details.  Once the version that can be installed via biocLite() is updated, I'll see if that provides any additional information.  If I can't get any useful feedback about the specific problem in Perl by updating my code, I'll send another reply on this thread.

Thanks,
Charles

-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 12:41 PM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel



----- Original Message -----
What happens if you just run the command above in isolation? 

Is T: a networked drive?
what does tempdir() display? Is it also a directory on drive T:? 
If so, try using a directory on a physical (local) drive as your output directory.

I'm not sure if the error code 2 that you are seeing is from perl or from R, but the googles (http://www.perlmonks.org/?node_id=274896) suggests that error 2 is "no such file or directory" so it could be a problem with one of the parameters you're passing to perl. 

BTW, I tried the following (forgive the bad line wrapping, that's what happens when I paste from windows into my mac):
[1] "E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl"
[1] "perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C:\\Users\\BI
OCBU~1\\AppData\\Local\\Temp\\2\\Rtmp4KS893/temp.txt C:\\Users\\BIOCBU~1\\AppData\\Local\\
Temp\\2\\Rtmp4KS893"
Uncaught exception from user code:
        Could not open C:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt! at E:/b
iocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35.
 at E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35
character(0)
attr(,"status")
[1] 2
Warning message:
running command 'perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C
:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt C:\Users\BIOCBU~1\AppData\Local\
Temp\2\Rtmp4KS893' had status 2
So this is complaining because the temp.txt file does not exist.

Henrik's suggestions are good too.

Dan
1 day later
#
Hi,

For those interested in this thread, it appears that the problem is caused by the library location.

Normally, R uses "C:\Users\cwarden\Documents\R\win-library", but R-devel uses "C:/Program Files/R/R-devel/library".

The problem is that the file paths are not provided to Perl in quotes.  Thus, the more specific error is "Can't open perl script "C:/Program": No such file or directory".  So, I should now be able to fix the problem.

Thanks for all of your help!

Best,
Charles

-----Original Message-----
From: Dan Tenenbaum [mailto:dtenenba at fhcrc.org] 
Sent: Monday, January 27, 2014 12:41 PM
To: Warden, Charles
Cc: bioc-devel at r-project.org
Subject: Re: [Bioc-devel] Problem running Perl script with R-devel



----- Original Message -----
What happens if you just run the command above in isolation? 

Is T: a networked drive?
what does tempdir() display? Is it also a directory on drive T:? 
If so, try using a directory on a physical (local) drive as your output directory.

I'm not sure if the error code 2 that you are seeing is from perl or from R, but the googles (http://www.perlmonks.org/?node_id=274896) suggests that error 2 is "no such file or directory" so it could be a problem with one of the parameters you're passing to perl. 

BTW, I tried the following (forgive the bad line wrapping, that's what happens when I paste from windows into my mac):
[1] "E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl"
[1] "perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C:\\Users\\BI
OCBU~1\\AppData\\Local\\Temp\\2\\Rtmp4KS893/temp.txt C:\\Users\\BIOCBU~1\\AppData\\Local\\
Temp\\2\\Rtmp4KS893"
Uncaught exception from user code:
        Could not open C:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt! at E:/b
iocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35.
 at E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl line 35
character(0)
attr(,"status")
[1] 2
Warning message:
running command 'perl E:/biocbld/bbs-2.14-bioc/R/library/COHCAP/Perl/create_wig_files.pl C
:\Users\BIOCBU~1\AppData\Local\Temp\2\Rtmp4KS893/temp.txt C:\Users\BIOCBU~1\AppData\Local\
Temp\2\Rtmp4KS893' had status 2
So this is complaining because the temp.txt file does not exist.

Henrik's suggestions are good too.

Dan
#
Hi Charles,

----- Original Message -----
I'm glad you figured it out. The location where R is installed doesn't change between release and devel versions, it's just an option you can set when installing R. 

Dan