LD_LIBRARY_PATH is harmfull

Consider the following situation: Solaris 8, /usr/local has gcc 2.95,
my home has gcc 3.4.3, my gcc is first on PATH, LD_LIBRARY_PATH is
unset (everything is perfect since I always use -R). In fact, programs
compiled with my gcc do not work if LD_LIBRARY_PATH is set to
something which has /usr/local/lib before home/lib, because it
overrides stored path (-R) and I got

  libgcc_s.so.1 (GCC_3.3) =>       (version not found)

I don't believe that happens unless `home/lib' is not in the library paths
at all.

I've checked man ld and man ld.so.1 on Solaris 8, and neither appear
to mention that -R is ignored if LD_LIBRARY_PATH is set.  I think
that is a (well-buried) Solaris-specific gotcha.

I found setting of LD_LIBRARY_PATH in bin/R (how /usr/local/lib get
into it at all and especially before PREFIX/lib??? -- there was no
LD_LIBRARY_PATH during configure!) and fixed it (although I spent
quite a while editing lib/R/bin/R and wondering an abscence of
any effect :-)

Please take a look at e.g. config.site, which explains this under LDFLAGS,
(as does the R-admin manual which INSTALL asks you to read).

After all the troubles I manage to load an extension, but, frankly, I
think there were too many problems. It would be very nice if R-project
reject the idea of using LD_LIBRARY_PATH because its setting is
considered harmfull and leads to too many problems:
http://www.linuxmafia.com/faq/Admin/ld-lib-path.html

[Which is Solaris specific, despite the site name.]

Unfortunately, the alternatives lead to even more problems, and this is
the first report we have had for years of a problem

(which can be solved on reading the documentation)

As the R-admin manual points out, we regularly test on Solaris 8 and
give an example there of setting LDFLAGS under the Solaris section.

Use of -R is harmful for sure!  It stops R being relocatable (so it
either could not be tested before being installed or it would not
run after installation),

and it is not at all portable.

Maybe one day when we have libtool tamed we will be able to use the
multiple equivalents of -R or LD_RUN_PATH in a portable-enough way.

Hi.

On Mon, 21 Feb 2005, Prof Brian Ripley wrote:

Consider the following situation: Solaris 8, /usr/local has gcc 2.95,
my home has gcc 3.4.3, my gcc is first on PATH, LD_LIBRARY_PATH is
unset (everything is perfect since I always use -R). In fact, programs
compiled with my gcc do not work if LD_LIBRARY_PATH is set to
something which has /usr/local/lib before home/lib, because it
overrides stored path (-R) and I got

  libgcc_s.so.1 (GCC_3.3) =>       (version not found)

I don't believe that happens unless `home/lib' is not in the library paths
at all.

Notice that I do have libgcc_s in /usr/local/lib, but it is for
gcc2.95, so ldd said `(version not found)' because there is a library
but with wrong version -- I guess other libraries simply are not
considered at all.

I've checked man ld and man ld.so.1 on Solaris 8, and neither appear
to mention that -R is ignored if LD_LIBRARY_PATH is set.  I think
that is a (well-buried) Solaris-specific gotcha.

The standard meaning of LD_LIBRARY_PATH is to be considered *before*
any other path.

I found setting of LD_LIBRARY_PATH in bin/R (how /usr/local/lib get
into it at all and especially before PREFIX/lib??? -- there was no
LD_LIBRARY_PATH during configure!) and fixed it (although I spent
quite a while editing lib/R/bin/R and wondering an abscence of
any effect :-)

Please take a look at e.g. config.site, which explains this under LDFLAGS,
(as does the R-admin manual which INSTALL asks you to read).

Note that I do not need any *additional* flags because
-L<home>/lib and -R<home>/lib are already in my specs file.

After all the troubles I manage to load an extension, but, frankly, I
think there were too many problems. It would be very nice if R-project
reject the idea of using LD_LIBRARY_PATH because its setting is
considered harmfull and leads to too many problems:
http://www.linuxmafia.com/faq/Admin/ld-lib-path.html

[Which is Solaris specific, despite the site name.]

AFAIK LD_LIBRARY_PATH has the same semantic on Linux and every other
recent *nix: it overrides system-wide defaults (e.g., crle on solaris
and ldconfig on linux) as well as embeded paths; -R or -rpath are also
common among *nixes.

Unfortunately, the alternatives lead to even more problems, and this is
the first report we have had for years of a problem

Note that I do not have a problem with R installation, but I was
troubled to load an extension (gmp in my case).

(which can be solved on reading the documentation)

Could you, please, enlight me what page of R-admin.pdf I missed?

As the R-admin manual points out, we regularly test on Solaris 8 and
give an example there of setting LDFLAGS under the Solaris section.

B.7.3 (p.22) says nothing about my case (32-bit solaris and gcc >3.3)

Use of -R is harmful for sure!  It stops R being relocatable (so it
either could not be tested before being installed or it would not
run after installation),

Testing before installtion is exactly the purpose of LD_LIBRARY_PATH:
you compile with -R for final installation and use
LD_LIBRARY_PATH=<build-dir>/lib during testing. Alternatively you can
use $ORIGIN so that libraries are searched in the places relative to
executable itself.

and it is not at all portable.

And could you name at least a single modern platform which supports
shared libraries and does not support an equivalent of -R for linking?

BTW, LD_LIBRARY_PATH also has different names on different *nix
(e.g., SHLIB_PATH on HP-UX, although AFAIK HP-UX also supports
LD_LIBRARY_PATH)

Maybe one day when we have libtool tamed we will be able to use the
multiple equivalents of -R or LD_RUN_PATH in a portable-enough way.

Nice to know that such plans exist :-)

  libgcc_s.so.1 (GCC_3.3) =>       (version not found)

I don't believe that happens unless `home/lib' is not in the library paths
at all.

Notice that I do have libgcc_s in /usr/local/lib, but it is for
gcc2.95, so ldd said `(version not found)' because there is a library
but with wrong version -- I guess other libraries simply are not
considered at all.

Something is very strange: notice you say you have 3.4.3 and it is looking
for GCC_3.3!

I get

         libgcc_s.so.1 =>         /usr/local/lib/libgcc_s.so.1

and no version number.

I've checked man ld and man ld.so.1 on Solaris 8, and neither appear
to mention that -R is ignored if LD_LIBRARY_PATH is set.  I think
that is a (well-buried) Solaris-specific gotcha.

The standard meaning of LD_LIBRARY_PATH is to be considered *before*
any other path.

Yes.  That is not what you say happens for you, though, rather you are
sayin *instead of*.

Note that I do not need any *additional* flags because
-L<home>/lib and -R<home>/lib are already in my specs file.

You need something to find R's shared libraries, e.g. libRlapack.

AFAIK LD_LIBRARY_PATH has the same semantic on Linux and every other
recent *nix: it overrides system-wide defaults (e.g., crle on solaris
and ldconfig on linux) as well as embeded paths; -R or -rpath are also
common among *nixes.

The issue is if they are ignored, not if they are overridden.

Could you, please, enlight me what page of R-admin.pdf I missed?

No, as I don't have your layout.  But it _is_ there:

If you have libraries and header files, e.g., for @acronym{GNU}
readline, in non-system directories, use the variables @code{LDFLAGS}
(for libraries, using @samp{-L} flags to be passed to the linker) and
@code{CPPFLAGS} (for header files, using @samp{-I} flags to be passed to
the C/C++ preprocessors), respectively, to specify these locations.
These default to @file{/usr/local/lib} and @file{/usr/local/include} to
catch the most common cases.

[You clearly do so have.]

Library paths specified as @option{-L/lib/path} in @code{LDFLAGS} are
collected together and prepended to @env{LD_LIBRARY_PATH} (or your
system's equivalent), so there should be no need for @option{-R} or
@option{-rpath} flags.

As the R-admin manual points out, we regularly test on Solaris 8 and
give an example there of setting LDFLAGS under the Solaris section.

B.7.3 (p.22) says nothing about my case (32-bit solaris and gcc >3.3)

It gives an example of setting LDFLAGS: you cannot expect every user's
setup to be mentioned.

And could you name at least a single modern platform which supports
shared libraries and does not support an equivalent of -R for linking?

Windows (probably the most-used platform for R).