[Bioc-devel] xps build problem on veracruz2

Dear Herve,

At the moment I found also the following link:
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jlab.org_theory_tag_cosyn_manual.pdf&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=7tn_6rjQ1Z2nt6iYVeE8untsO9LEqi2ckK_BvP73l2s&s=Ui9eBMvQBWYFJYpwM5Jjrn_3Hjb6spCYMlb0BbdERhs&e=
which states:
Error Wrong ELF class.  If you get errors mentioning ELF-class then you
are probably linking 64 to 32 bit code which does not compute. Make sure
you are linking to the 64 bit libraries of root and not the 32 bit ones.
For me the 64 bit libraries were located in /usr/local/lib/root while
the 32 bit ones were in /usr/local/root/lib/root.

I am not sure what this means and if it is helpful for you.

Best regards,
Christian


On 04/22/17 23:57, cstrato wrote:

Dear Herve,

Thank you for your efforts to get xps running.

I will make the changes in Makefile.arch, too, and update README and
rename it to install.

I am also surprised that the libs get installed under <prefix>/lib/root,
and I will check if there is a reason for this.

As far as I remember, long time ago someone had a similar problem, see
e.g.:
https://urldefense.proofpoint.com/v2/url?u=https-3A__support.bioconductor.org_p_51838_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=7tn_6rjQ1Z2nt6iYVeE8untsO9LEqi2ckK_BvP73l2s&s=31gB3jOjHIBy36ASjb9hE-_OU7g_zJtAFZOwYYrU9EY&e=

Do you use the environment variable ROOTSYS?

Best regards,
Christian



On 04/22/17 22:36, Herv? Pag?s wrote:

Hi Christian,

Thanks for the update. Glad it works for you.

One small thing is that, if CMAKE_INSTALL_PREFIX is specified
when configuring ROOT, the ROOT libs get installed under
<prefix>/lib/root, not under <prefix>/lib. I was surprised by
this, but that's what I got when I installed ROOT on veracruz2.
I configured with:

    export CC=/usr/local/clang4/bin/clang
    export CXX=/usr/local/clang4/bin/clang++
    cmake -DCMAKE_INSTALL_PREFIX=/usr/local/root -Dgnuinstall=ON
-Dfortran=OFF -Dmysql=OFF -Dsqlite=OFF ../root

Then built with:

    cmake --build . -- -j4

Then installed with:

    sudo cmake --build . --target install

And the libraries got installed under /usr/local/root/lib/root

So when trying to install the latest xps, loading xps.so failed for
me because the ROOT libraries were not found. The following change
to Makefile.arch fixed the problem:

$ svn diff Makefile.arch
Index: Makefile.arch
===================================================================
--- Makefile.arch    (revision 129046)
+++ Makefile.arch    (working copy)
@@ -261,7 +261,7 @@
 CXXFLAGS      = $(OPT2) -pipe -Wall -W -Woverloaded-virtual
 LD            = $(MACOSXTARGET) g++
 #LDFLAGS       = $(OPT2)
-LDFLAGS       = $(OPT2) -rpath $(shell $(ROOTCONFIG) --prefix)/lib
+LDFLAGS       = $(OPT2) -rpath $(shell $(ROOTCONFIG) --prefix)/lib
-rpath $(shell $(ROOTCONFIG) --prefix)/lib/root
 UNDEFOPT      = dynamic_lookup
 # The SOFLAGS will be used to create the .dylib,
 # the .so will be created separately

Also note that the rpaths specified at linking time get hardcoded in
xps.so:

veracruz2:src biocbuild$ otool -l xps.so | tail -n 18
Load command 31
          cmd LC_RPATH
      cmdsize 32
         path /usr/local/root/lib (offset 12)
Load command 32
          cmd LC_RPATH
      cmdsize 40
         path /usr/local/root/lib/root (offset 12)
Load command 33
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 4141784
 datasize 14336
Load command 34
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 4156120
 datasize 904

So the end user will need to have the ROOT libraries at these locations
too (unless s/he installs from source of course). This would need to be
explained in xps README file (which BTW should preferably be named
INSTALL).

Thanks,
H.


On 04/22/2017 05:15 AM, cstrato wrote:

Dear Herve,

I am glad to inform you that I have just uploaded version xps_1.35.3 to
BioC-dev branch. I have followed your suggestion to use -rpath and have
eliminated the environment variables DYLD_LIBRARY_PATH and
LD_LIBRARY_PATH.

I have tested the new version on both Yosemite and on Sierra with
csrutil enabled! Thus I assume that it will also run on El Capitan.

Best regards,
Christian

P.S.:
Please allow me to comment on your note on [DY]LD_LIBRARY_PATH.
As you know xps was uploaded to Bioc 10 years ago (with your kind help)
and is available on BioC since 9 years.
At that time the environment variables [DY]LD_LIBRARY_PATH were
necessary, and for many years required by ROOT. Since xps did run on
the
Mac on all systems from Leopard till Yosemite w/o problems I had no
need
to change it.
Furthermore, I had not heard that the use of these variables have been
discouraged, just like many other developers who only now realize that
they have to use rpath or simply disable csrutil (I have realized this
when googling around).



On 04/21/17 00:29, Herv? Pag?s wrote:

Also relying on [DY]LD_LIBRARY_PATH is considered bad practice and
has been discouraged for years. xps is the only Bioconductor package
that relies on these variables for its configure/build process.

H.

On 04/20/2017 03:24 PM, Dan Tenenbaum wrote:

Disabling SIP should not be done anywhere. Every page I've read on
this topic strongly discourages doing this.


----- Original Message -----

From: "Herv? Pag?s" <hpages at fredhutch.org>
To: "cstrato" <cstrato at aon.at>, "Dan Tenenbaum"
<dtenenba at fredhutch.org>
Cc: "bioc-devel" <bioc-devel at r-project.org>
Sent: Thursday, April 20, 2017 3:17:23 PM
Subject: Re: [Bioc-devel] xps build problem on veracruz2

On 04/20/2017 03:01 PM, cstrato wrote:

Dear Herve,

Doing 'csrutil disable' does indeed solve the problem, since:

1, in this way I was able to build xps on Mac OS Sierra

2, in this way I could already help one user of xps, see:
https://urldefense.proofpoint.com/v2/url?u=https-3A__support.bioconductor.org_p_90056_-2390247&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q0hKDI_veZEYmjrYbsK1Xqu--fGdfl_JmfSKLygl_dg&s=GfzgU1Ibm_scFWO58Mv_ZfxKtn-FSJgkxMW1ZBYK1Vs&e=






This means, that users of xps seem to be willing to disable
csrutil.

I'm not a statistician but I wouldn't draw conclusions from a sample
of size 1. Mac users who cannot install xps on their machine will
use something else, or, if they desperately need xps, they will
grab a Linux box.

Sorry but disabling SIP on our Mac builders is not an option.

However, I just realized that there may be an even greater problem,
namely, which version of ROOT should a user install?

People can download ROOT binaries built with XCode 7.x from:
https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_content_release-2D53436&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q0hKDI_veZEYmjrYbsK1Xqu--fGdfl_JmfSKLygl_dg&s=LByiYPyOfsHnRc9oOqXOs9xMcfltktXgkTQ3sh5hKFc&e=





for OS X 10.10 and 10.11.

Thus for El Capitan they can download the following binaries:
- root_v5.34.36.macosx64-10.11-clang70.dmg
- root_v5.34.36.macosx64-10.11-clang70.tar.gz

Did you install one of those binaries or did you compile ROOT from
source?

As I said earlier, I compiled ROOT 5 from source on veracruz2.

With the help of my README file people could compile ROOT from
source
for XCode 8.x.


However, you have mentioned that the CRAN people are using clang
4.0.0
for producing the Mac binaries of R and CRAN packages and thus you
are
using the same on veracruz2.

Yes.

Did you compile ROOT with XCode 7 or 8 or did you use clang 4.0.0,
which
is not officially supported by Apple?

With clang 4.0.0.

The question is whether xps built with this version of ROOT will
work
with the ROOT binaries which people can download from ROOT?

I guess someone will need to figure this out.

Note that if people need to compile their own ROOT anyway in
order to
be able to use the xps binary we distribute, then they should
also be
able to install xps from source. So that defeats the purpose of
providing a binary in the first place.

Cheers,
H.

Best regards,
Christian



On 04/20/17 20:00, Herv? Pag?s wrote:

On 04/20/2017 10:59 AM, Herv? Pag?s wrote:

Hi Christian,

Disabling 'csrutil disable' might help xps on veracruz2 but that

  ^^^^^^^^^^^^^^^^^^^^^^^^^^
oops, no double negative intended here. I meant, doing 'csrutil
disable'
might help... etc

H.

won't help your end users.

I'm no expert in developing a package on Mac but other people on
this list are. Also R-SIG-Mac might be a good place to seek help
for this.

Cheers,
H.


On 04/20/2017 10:53 AM, cstrato wrote:

Dear Herve,

Thank you for your efforts to try to install xps. I am glad to
hear
that
you could build ROOT 5 from source.

It's a pity that Apple does no longer allow the use of
DYLD_LIBRARY_PATH. This seems to break the code of a lot of
people
(when
googling around).

I will try to change the build process and will see if I
succeed.
However, at the moment I have a couple of questions:


1, Is there any reason that you do not want to 'csrutil
disable'?


2, It is easy to delete the test for DYLD_LIBRARY_PATH in my
'configure.in' file, however, I have also to change the
'Makefile'.

Since you say that the problem can be solved by adding the
-rpath
flag
when linking, I assume that I have to change  the following line
in my
'Makefile':
        $(LD) -bundle $(LDFLAGS) $^ $(GLIBS) $(MYLIBS) \
           $(OutPutOpt) $(subst .$(DllSuf),.so,$@)

Since I am not familiar with -rpath can you give me a hint
how to
change
it?


3, There is a Wikipedia explanation for '-rpath', see:
https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Rpath&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=ge_d4eBoKDAggNWVQUzMVPAJy250VlZuPTXcPyr20HM&e=









Interestingly, there is the following line:

'Instead of specifying the -rpath to the linker, the environment
variable LD_RUN_PATH can be set to the same effect.'

Do you think that using LD_RUN_PATH would solve the problem?
If yes, then how?


4, Googling for the DYLD_LIBRARY_PATH problem I have also found
the
following discussion:
https://urldefense.proofpoint.com/v2/url?u=https-3A__forums.macrumors.com_threads_is-2Dit-2Dok-2Dto-2Duse-2Ddyld-5Flibrary-5Fpath-2Don-2Dmac-2Dos-2Dx-2Dand-2Dwhat-25C2-2592s-2Dthe-2Ddynamic-2Dlibrary-2Dsearch.956258_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=yhLXARIG_RS1LNegc7tbjO1bayLOp7zy5-rzbtjCHIk&e=









There, one answer mentioned:
'Also, rpath is a good idea. Also see install_name_tool, which
can
change the load paths of libraries.'

Doing a search for 'install_name_tool' I found:
https://urldefense.proofpoint.com/v2/url?u=http-3A__stackoverflow.com_questions_2985315_using-2Dinstall-2Dname-2Dtool-2Dwhats-2Dgoing-2Dwrong&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=r5LiHlxkGAeJAcciwteD2XD6ffNtWLiknvcIj9EJL8E&e=









There, one answer mentioned (see also $man install_name_tool):
'Having experimented more: install_name_tool -id newname file
will do
the trick.'

See also:
https://urldefense.proofpoint.com/v2/url?u=http-3A__qin.laya.com_tech-5Fcoding-5Fhelp_dylib-5Flinking.html&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=FL17xv1uuzR6UF59mgJstMQ-1seE1UeKDQVVXQgXnUo&e=










Do you think that either using the environment variable
LD_RUN_PATH or
using 'install_name_tool' could solve the problem without having
to use
'-rpath'?


Thank you in advance.
Best regards,
Christian



On 04/19/17 22:51, Herv? Pag?s wrote:

Hi Christian,

So I installed ROOT 5 from source on veracruz2. It's in
/usr/local/root.

However, Apple's SIP (System Integrity Protection, new and
enabled by default on El Capitan) is getting in the way when
trying to install xps. That's because xps configure and build
process relies on DYLD_LIBRARY_PATH. Problem is that this
environment variable (and any other variables that control
dynamic loading) is not inherited by child processes when SIP
is on:

veracruz2:~ biocbuild$ if test "${DYLD_LIBRARY_PATH}"; then
echo
'yep!';
else echo 'nope!'; fi
yep!

veracruz2:~ biocbuild$ sh
sh-3.2$ if test "${DYLD_LIBRARY_PATH}"; then echo 'yep!'; else
echo
'nope!'; fi
nope!

That breaks xps configure script:

veracruz2:~ biocbuild$ export
LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH

veracruz2:~ biocbuild$ echo $LD_LIBRARY_PATH
/usr/local/mysql/lib:/usr/local/root/lib/root:/ImageMagick-7.0.5/lib:/usr/local/ensembl-vep/htslib











veracruz2:~ biocbuild$ R CMD INSTALL xps
* installing to library
?/Library/Frameworks/R.framework/Versions/3.4/Resources/library?

* installing *source* package ?xps? ...
checking for gcc... clang
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ANSI C... none needed
checking how to run the C preprocessor... clang -E
checking for gcc... (cached) clang
checking whether we are using the GNU C compiler... (cached)
yes
checking whether clang accepts -g... (cached) yes
checking for clang option to accept ANSI C... (cached) none
needed
found ROOT version 5.34/36 in directory /usr/local/root

xps configuration error:

   You must set the shell variable LD_LIBRARY_PATH to the
   directory where ROOT resides and re-run R CMD INSTALL
   e.g., (using Bourne shell syntax):

      export "LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH"
      R CMD INSTALL xps

   Please consult the README file for more information

ERROR: configuration failed for package ?xps?
* removing
?/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps?





* restoring previous
?/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps?






That also breaks the dynlib mechanism because, after I
managed to
produce xps.so, it turns out that this shared object is
linked to
the ROOT libraries via the @rpath mechanism:

veracruz2:src biocbuild$ otool -L xps.so
xps.so:
    xps.so (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libGui.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libCore.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libCint.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libRIO.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libNet.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libHist.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libGraf.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libGraf3d.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libGpad.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libTree.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libRint.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libPostscript.so (compatibility version 0.0.0,
current
version 0.0.0)
    @rpath/libMatrix.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libPhysics.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libMathCore.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libThread.so (compatibility version 0.0.0, current
version
0.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
current
version 1226.10.1)
    @rpath/libGed.so (compatibility version 0.0.0, current
version
0.0.0)
    @rpath/libTreePlayer.so (compatibility version 0.0.0,
current
version 0.0.0)
    @rpath/libTreeViewer.so (compatibility version 0.0.0,
current
version 0.0.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0,
current
version 120.1.0)

so it can't be loaded in R:

dyn.load("xps.so")

Error in dyn.load("xps.so") :
  unable to load shared object
'/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so':
  dlopen(/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so, 6):
Library not
loaded: @rpath/libGui.so
  Referenced from: /Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so
  Reason: image not found

because R has no access to DYLD_LIBRARY_PATH:

Sys.getenv("DYLD_LIBRARY_PATH")

[1] ""

This can be addressed by adding the following flag when
linking:

  -rpath $(shell $(ROOTCONFIG) --prefix)/lib

Do you think you can revisit xps configure and build process?
Make
sure
you test it on a machine where SIP is enabled.

Thanks,
H.

 snip >>>

On 03/23/17 17:47, Herv? Pag?s wrote:

Hi Christian,

The CRAN folks are currently experimenting with
clang
4.0.0
for
producing the Mac binaries of R and CRAN packages so
we are
using
the same on veracruz2. This is a version of clang
that is
ahead of
what's in XCode 8.x or XCode 7.x. So I guess that
means
we'll
have
to compile ROOT from source on veracruz2.

BTW any reason not to make xps work with ROOT 6?

Cheers,
H.

On 03/23/2017 07:28 AM, cstrato wrote:

Dear Valerie,

I have seen that you have set up a new Mac server,
veracruz2,
running El
Capitan.

Although the development version of xps does even
run on
Mac OS
Sierra,
one issue still remains the same:

You need to install the latest ROOT version 5,
since
xps
does
not
run
with ROOT 6!

So you need to install on veracruz2 the same root
version
that
you
have
installed on toluca2 running Maverics, i.e.
root_v5.34.36.macosx64-10.11-clang70.dmg

However, if you have installed on El Capitan XCode
8.x
instead of
XCode
7.x, then you need to compile ROOT from source,
i.e.:
https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_download_root-5Fv5.34.36.source.tar.gz&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q9mk6yIytaNZlSdiLX_dFwchX8Tb7ra6x3WBBNIcs2o&s=Lz7YkqZ3XwjRsYIXVTbSvbDvTM-jTyoWvoVSa1PdBDw&e=












The README file of xps does explain how to compile
ROOT
for
Sierra. This
should also be valid for El Capitan running XCode
8.x.

Thank you in advance.
Best regards,
Christian
_._._._._._._._._._._._._._._._._._
C.h.r.i.s.t.i.a.n   S.t.r.a.t.o.w.a
V.i.e.n.n.a           A.u.s.t.r.i.a
e.m.a.i.l:        cstrato at aon.at
_._._._._._._._._._._._._._._._._._

--
Herv? Pag?s

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319

--
Herv? Pag?s

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319