L.S.
In this bug report
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14493#c1
it is mentioned that after moving an R installation
one should regenerate the Rscript executable.
Is there an easy way to do so (after an R installation has been
moved)?
I have not found any information in the R installation and
administration manual.
Many thanks in advance for any pointer.
Best wishes,
Tobias
P.S. The background to this question is the usage of Rscript
calls in the Makevars files of some R packages on CRAN, so
the 'broken' Rscript prevents installation of certain R packages.
Tobias Verbeke
Manager
OpenAnalytics BVBA
Jupiterstraat 20
2600 Antwerp
Belgium
E tobias.verbeke at openanalytics.eu
M +32 499 36 33 15
http://www.openanalytics.eu
On 21 September 2013 at 20:43, Tobias Verbeke wrote:
| P.S. The background to this question is the usage of Rscript
| calls in the Makevars files of some R packages on CRAN, so
| the 'broken' Rscript prevents installation of certain R packages.
More details, please.
AFAICT there is no 'broken' Rscript per se. Eg for Rcpp, and per hints from
Kurt el al over the years, we have been doing this for a few years
PKG_CXXFLAGS=-I../inst/include
PKG_LIBS=`$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"` $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
where a key part is the `$(R_HOME)/bin` which permits you to transparently
switch between R-release, R-devel, R-beforeMove, R-afterMove, R-whatevr, ...
simply by adjusting your shell's $PATH variable, or the R wrapper you for R
CMD, or ...
It. Just. Works.
So if you borked your Rscript somewhere, just use another and quickly rebuild
R for the new location. It is after all a cheap build (and cheaper still if
you use tricks like ccache which I am a huge fan of, or 'make -j8', or ...)
Dirk
PS ccache is at http://ccache.samba.org/ and should be in any sane Linux distro
Hi Dirk,
Many thanks for your reaction.
----- Original Message -----
From: "Dirk Eddelbuettel" <edd at debian.org>
To: "Tobias Verbeke" <tobias.verbeke at openanalytics.eu>
Cc: r-devel at r-project.org
Sent: Saturday, September 21, 2013 9:00:12 PM
Subject: Re: [Rd] regenerate Rscript after moving R installation
On 21 September 2013 at 20:43, Tobias Verbeke wrote:
| P.S. The background to this question is the usage of Rscript
| calls in the Makevars files of some R packages on CRAN, so
| the 'broken' Rscript prevents installation of certain R packages.
More details, please.
AFAICT there is no 'broken' Rscript per se. Eg for Rcpp, and per hints from
Kurt el al over the years, we have been doing this for a few years
PKG_CXXFLAGS=-I../inst/include
PKG_LIBS=`$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"` $(LAPACK_LIBS)
$(BLAS_LIBS) $(FLIBS)
The package that made me discover this was RcppEigen which has indeed
PKG_LIBS=`$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"` $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
where a key part is the `$(R_HOME)/bin` which permits you to transparently
switch between R-release, R-devel, R-beforeMove, R-afterMove, R-whatevr, ...
simply by adjusting your shell's $PATH variable, or the R wrapper you for R
CMD, or ...
It. Just. Works.
It is neat and certainly works, unless R is built on another location (on a build machine)
prior to being put on its final location.
If I read the strace output below correctly, the origin of the problem is the hardcoded
location of the R binary.
tobias at oa-laptop:/opt/architect/architect-stable/plugins/eu.openanalytics.architect.r.server.gtk.linux_stable/R/bin$ strace ./Rscript -e '2+2'
execve("./Rscript", ["./Rscript", "-e", "2+2"], [/* 51 vars */]) = 0
brk(0) = 0x1eb6000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc7563e1000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=161604, ...}) = 0
mmap(NULL, 161604, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc7563b9000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\30\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1811128, ...}) = 0
mmap(NULL, 3925208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc755e02000
mprotect(0x7fc755fb7000, 2093056, PROT_NONE) = 0
mmap(0x7fc7561b6000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b4000) = 0x7fc7561b6000
mmap(0x7fc7561bc000, 17624, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc7561bc000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc7563b8000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc7563b7000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc7563b6000
arch_prctl(ARCH_SET_FS, 0x7fc7563b7700) = 0
mprotect(0x7fc7561b6000, 16384, PROT_READ) = 0
mprotect(0x601000, 4096, PROT_READ) = 0
mprotect(0x7fc7563e3000, 4096, PROT_READ) = 0
munmap(0x7fc7563b9000, 161604) = 0
brk(0) = 0x1eb6000
brk(0x1ed7000) = 0x1ed7000
execve("/home/builduser/architect/stable/R-build/sources/R-3.0.1/bin/R", ["/home/builduser/architect/stable/"..., "--slave", "--no-restore", "-e", "2+2", "--args"], [/* 52 vars */]) = -1 ENOENT (No such file or directory)
dup(2) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc7563e0000
lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(3, "Rscript execution error: No such"..., 51Rscript execution error: No such file or directory
) = 51
close(3) = 0
munmap(0x7fc7563e0000, 4096) = 0
exit_group(-1) = ?
So if you borked your Rscript somewhere, just use another and quickly rebuild
R for the new location. It is after all a cheap build (and cheaper still if
you use tricks like ccache which I am a huge fan of, or 'make -j8', or ...)
There is no way to build again since R itself is shipped (in a Debian/Ubuntu package
and as part of Architect) prior to being installed and used on another computer.
The buildstamp that is part of the final installation path of the application is
generated after the R build (since R is only one component), so changing the --prefix
on the build machine would not work (currently).
I hope this gives more background to the question and would be curious if there are alternatives
to rapidly regenerate the Rscript executable only.
(My other alternative of messing with the path in a hex editor has not been successful :-)
Best wishes,
Tobias
P.S. Architect can be installed from
http://deb.openanalytics.eu/howto.html
sudo apt-get install architect
On 21 September 2013 at 21:39, Tobias Verbeke wrote:
| The package that made me discover this was RcppEigen which has indeed
Never heard of it :)
| PKG_LIBS=`$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"` $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
|
| > where a key part is the `$(R_HOME)/bin` which permits you to transparently
| > switch between R-release, R-devel, R-beforeMove, R-afterMove, R-whatevr, ...
| > simply by adjusting your shell's $PATH variable, or the R wrapper you for R
| > CMD, or ...
| >
| > It. Just. Works.
|
| It is neat and certainly works, unless R is built on another location (on a build machine)
| prior to being put on its final location.
|
| If I read the strace output below correctly, the origin of the problem is the hardcoded
| location of the R binary.
[...]
| There is no way to build again since R itself is shipped (in a Debian/Ubuntu package
| and as part of Architect) prior to being installed and used on another computer.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I do not know what 'Architekt' is -- but maybe you just mistakenly assume
that you can mv(1) installations at will? And maybe you can't? Consider the
configure --prefix=/some/location/ ...
make
make install
where /some/location gets set at compile-time.
You seem to wish it didn't. But wishing alone may not make it so. In all
seriousness, these binaries may not be relocatable at will.
| The buildstamp that is part of the final installation path of the application is
| generated after the R build (since R is only one component), so changing the --prefix
| on the build machine would not work (currently).
|
| I hope this gives more background to the question and would be curious if there are alternatives
| to rapidly regenerate the Rscript executable only.
|
| (My other alternative of messing with the path in a hex editor has not been successful :-)
I'd go back to rebuilding for the target location.
Dirk
From: "Dirk Eddelbuettel" <edd at debian.org>
To: "Tobias Verbeke" <tobias.verbeke at openanalytics.eu>
Cc: "Dirk Eddelbuettel" <edd at debian.org>, r-devel at r-project.org
Sent: Saturday, September 21, 2013 11:03:37 PM
Subject: Re: [Rd] regenerate Rscript after moving R installation
Tobias,
On 21 September 2013 at 21:39, Tobias Verbeke wrote:
| The package that made me discover this was RcppEigen which has indeed
Never heard of it :)
| PKG_LIBS=`$(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()"` $(LAPACK_LIBS)
| $(BLAS_LIBS) $(FLIBS)
|
| > where a key part is the `$(R_HOME)/bin` which permits you to
| > transparently
| > switch between R-release, R-devel, R-beforeMove, R-afterMove, R-whatevr,
| > ...
| > simply by adjusting your shell's $PATH variable, or the R wrapper you for
| > R
| > CMD, or ...
| >
| > It. Just. Works.
|
| It is neat and certainly works, unless R is built on another location (on a
| build machine)
| prior to being put on its final location.
|
| If I read the strace output below correctly, the origin of the problem is
| the hardcoded
| location of the R binary.
[...]
| There is no way to build again since R itself is shipped (in a
| Debian/Ubuntu package
| and as part of Architect) prior to being installed and used on another
| computer.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I do not know what 'Architekt' is -- but maybe you just mistakenly assume
Architect is a cross-platform Eclipse based R IDE with a.o. out-of-the-box Rcpp support ;-)
that you can mv(1) installations at will? And maybe you can't? Consider the
configure --prefix=/some/location/ ...
make
make install
where /some/location gets set at compile-time.
You seem to wish it didn't. But wishing alone may not make it so. In all
seriousness, these binaries may not be relocatable at will.
That is fair enough. I do not expect R-devel to be populated with Djinns,
but was puzzled (and probably lead down the wrong path) by the fact that
Rscript was the only stubborn piece in our build procedures.
| The buildstamp that is part of the final installation path of the
| application is
| generated after the R build (since R is only one component), so changing
| the --prefix
| on the build machine would not work (currently).
|
| I hope this gives more background to the question and would be curious if
| there are alternatives
| to rapidly regenerate the Rscript executable only.
|
| (My other alternative of messing with the path in a hex editor has not been
| successful :-)
I'd go back to rebuilding for the target location.
Thank you! I will report back if I find out more.
Best,
Tobias
On Sep 21, 2013, at 8:43 PM, Tobias Verbeke <tobias.verbeke at openanalytics.eu> wrote:
L.S.
In this bug report
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14493#c1
it is mentioned that after moving an R installation
one should regenerate the Rscript executable.
Is there an easy way to do so (after an R installation has been
moved)?
You cannot move installed R. Once you run make install, there are several places in which paths get baked in - mainly Rscript and the R start script. What I typically do for deployment on the Labs machines is to use make install rhome=<xxx> where <xxx> is some path that I can always create a symlink in (I also use DESTDIR so that path doesn't actually need to exist on the build machine and it avoid polluting --prefix which is not needed). That way you can move R wherever you want as long so you keep that one symlink up to date.
Cheers,
Simon
I have not found any information in the R installation and
administration manual.
Many thanks in advance for any pointer.
Best wishes,
Tobias
P.S. The background to this question is the usage of Rscript
calls in the Makevars files of some R packages on CRAN, so
the 'broken' Rscript prevents installation of certain R packages.
--
Tobias Verbeke
Manager
OpenAnalytics BVBA
Jupiterstraat 20
2600 Antwerp
Belgium
E tobias.verbeke at openanalytics.eu
M +32 499 36 33 15
http://www.openanalytics.eu
I forgot to mention that some packages bake-in paths as well, so even if you fix both R and Rscript, it will still not work in general.
On Sep 22, 2013, at 3:42 AM, Simon Urbanek <simon.urbanek at r-project.org> wrote:
On Sep 21, 2013, at 8:43 PM, Tobias Verbeke <tobias.verbeke at openanalytics.eu> wrote:
L.S.
In this bug report
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14493#c1
it is mentioned that after moving an R installation
one should regenerate the Rscript executable.
Is there an easy way to do so (after an R installation has been
moved)?
You cannot move installed R. Once you run make install, there are several places in which paths get baked in - mainly Rscript and the R start script. What I typically do for deployment on the Labs machines is to use make install rhome=<xxx> where <xxx> is some path that I can always create a symlink in (I also use DESTDIR so that path doesn't actually need to exist on the build machine and it avoid polluting --prefix which is not needed). That way you can move R wherever you want as long so you keep that one symlink up to date.
Cheers,
Simon
I have not found any information in the R installation and
administration manual.
Many thanks in advance for any pointer.
Best wishes,
Tobias
P.S. The background to this question is the usage of Rscript
calls in the Makevars files of some R packages on CRAN, so
the 'broken' Rscript prevents installation of certain R packages.
--
Tobias Verbeke
Manager
OpenAnalytics BVBA
Jupiterstraat 20
2600 Antwerp
Belgium
E tobias.verbeke at openanalytics.eu
M +32 499 36 33 15
http://www.openanalytics.eu
Dear Simon,
Many thanks for all additional detail and insight into
your practice using rhome and symlinks. Much appreciated!
Best,
Tobias
----- Original Message -----
From: "Simon Urbanek" <simon.urbanek at r-project.org>
To: "Tobias Verbeke" <tobias.verbeke at openanalytics.eu>
Cc: r-devel at r-project.org
Sent: Sunday, September 22, 2013 3:45:41 AM
Subject: Re: [Rd] regenerate Rscript after moving R installation
I forgot to mention that some packages bake-in paths as well, so even if you
fix both R and Rscript, it will still not work in general.
On Sep 22, 2013, at 3:42 AM, Simon Urbanek <simon.urbanek at r-project.org>
wrote:
On Sep 21, 2013, at 8:43 PM, Tobias Verbeke
<tobias.verbeke at openanalytics.eu> wrote:
L.S.
In this bug report
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14493#c1
it is mentioned that after moving an R installation
one should regenerate the Rscript executable.
Is there an easy way to do so (after an R installation has been
moved)?
You cannot move installed R. Once you run make install, there are several
places in which paths get baked in - mainly Rscript and the R start
script. What I typically do for deployment on the Labs machines is to use
make install rhome=<xxx> where <xxx> is some path that I can always create
a symlink in (I also use DESTDIR so that path doesn't actually need to
exist on the build machine and it avoid polluting --prefix which is not
needed). That way you can move R wherever you want as long so you keep
that one symlink up to date.
Cheers,
Simon
I have not found any information in the R installation and
administration manual.
Many thanks in advance for any pointer.
Best wishes,
Tobias
P.S. The background to this question is the usage of Rscript
calls in the Makevars files of some R packages on CRAN, so
the 'broken' Rscript prevents installation of certain R packages.
--
Tobias Verbeke
Manager
OpenAnalytics BVBA
Jupiterstraat 20
2600 Antwerp
Belgium
E tobias.verbeke at openanalytics.eu
M +32 499 36 33 15
http://www.openanalytics.eu
On Sep 24, 2013, at 2:18 AM, Murray Stokely <murray at stokely.org> wrote:
Simon, do you have some examples of packages with this attribute?
As Brian pointed out, some of them concern linking (I'm typically worried about OS X and Dirk has fixed Rcpp there), but there are also others that store package paths in variables or in supplementary files. Others store system paths as well (i.e. you can move R but not any dependent resources). Unfortunately this is hard to detect, because of lazy-loading (all lazy-load files have the paths baked in the namespace, so they will always include the path that you installed to originally) -- one example of a hard-coded variable is race.
Removing the hard-coding of paths in base R and Rscript is one of the many local patches we've maintained in the R I use at my workplace since at least the R 2.5 days. We do this to enable us to send R and all its dependencies off to build farms, unit test clusters, and production clusters for running parallel computations among other use cases where the path of the build server is irrelevant to the server running the R code.
I don't recall running into any packages where an absolute path from the build host was hard-coded into the package such that we had to update code to get it to work. But maybe I'm just not using those packages.
It is hard to detect. The linking ones typically fail right away, but those with hard-coded paths inside may fail only in some use cases.
Cheers,
Simon
- Murray
On Sat, Sep 21, 2013 at 6:45 PM, Simon Urbanek <simon.urbanek at r-project.org> wrote:
I forgot to mention that some packages bake-in paths as well, so even if you fix both R and Rscript, it will still not work in general.
On Sep 22, 2013, at 3:42 AM, Simon Urbanek <simon.urbanek at r-project.org> wrote:
On Sep 21, 2013, at 8:43 PM, Tobias Verbeke <tobias.verbeke at openanalytics.eu> wrote:
L.S.
In this bug report
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14493#c1
it is mentioned that after moving an R installation
one should regenerate the Rscript executable.
Is there an easy way to do so (after an R installation has been
moved)?
You cannot move installed R. Once you run make install, there are several places in which paths get baked in - mainly Rscript and the R start script. What I typically do for deployment on the Labs machines is to use make install rhome=<xxx> where <xxx> is some path that I can always create a symlink in (I also use DESTDIR so that path doesn't actually need to exist on the build machine and it avoid polluting --prefix which is not needed). That way you can move R wherever you want as long so you keep that one symlink up to date.
Cheers,
Simon
I have not found any information in the R installation and
administration manual.
Many thanks in advance for any pointer.
Best wishes,
Tobias
P.S. The background to this question is the usage of Rscript
calls in the Makevars files of some R packages on CRAN, so
the 'broken' Rscript prevents installation of certain R packages.
--
Tobias Verbeke
Manager
OpenAnalytics BVBA
Jupiterstraat 20
2600 Antwerp
Belgium
E tobias.verbeke at openanalytics.eu
M +32 499 36 33 15
http://www.openanalytics.eu