compile question

Dear all,

Starting all over again with R-devel of 22-11-2015.

After getting the dependancies (minimal graphics initially) I have 
been able to come this far - with a 64-bit build on AIX.



This is my configure call:

#!/usr/bin/ksh
# my_config_test

. /data/prj/cran/64/setup.env

./configure --enable-maintainer-mode  --enable-R-shlib --enable-lto 
--with-libpth-prefix=prefix=/opt \

--prefix=/opt \

        --disable-rpath --with-readline=no --with-x=no \
        --with-cairo=no --with-libpng=no --with-jpeglib=no 
--with-libtiff=no \
        --with-aix-soname=aix

So, modified the configure call, just incase, to be a bit more 'normal', 
but still getting a failure with:

creating src/scripts/R.fe
make[3]: Entering directory '/data/prj/cran/64/R-devel/src/scripts'
mkdir -p -- ../../bin
make[3]: Leaving directory '/data/prj/cran/64/R-devel/src/scripts'
make[2]: Leaving directory '/data/prj/cran/64/R-devel/src/scripts'
make[2]: Entering directory '/data/prj/cran/64/R-devel/src/include'
mkdir -p -- ../../include
/bin/sh: ../../tools/missing:  not found.
Makefile:66: recipe for target 'stamp-h.in' failed
make[2]: *** [stamp-h.in] Error 127
make[2]: Leaving directory '/data/prj/cran/64/R-devel/src/include'
Makefile:28: recipe for target 'R' failed
make[1]: *** [R] Error 1
make[1]: Leaving directory '/data/prj/cran/64/R-devel/src'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1

Everything seems top be in place, but I do not know GNU autotools well 
enough (I thought that is the origin of 'missing')

root at x072:[/data/prj/cran/64/R-devel]cd src/include
root at x072:[/data/prj/cran/64/R-devel/src/include]find ../.. -name missing
../../tools/missing
root at x072:[/data/prj/cran/64/R-devel/src/include]find ../.. -name 
missing -ls
101915983   11 -rwxrwxr-x  1 2300     2300        11135 Nov 14  2013 
../../tools/missing
root at x072:[/data/prj/cran/64/R-devel/src/include]head  ../../tools/missing
#! /bin/sh
# Common stub for a few missing GNU programs while installing.

scriptversion=2006-05-10.23

# Copyright (C) 1996, 1997, 1999, 2000, 2002, 2003, 2004, 2005, 2006
#   Free Software Foundation, Inc.
# Originally by Fran,cois Pinard <pinard at iro.umontreal.ca>, 1996.

# This program is free software; you can redistribute it and/or modify

And the environment is:
root at x072:[/data/prj/cran/64/R-devel]cat ../setup*
export OBJECT_MODE=64
export CC="gcc -maix64"
export CXX="g++ -maix64"
export F77="gfortran -maix64"
export FC="gfortran -maix64"
export LDFLAGS="-L/opt/lib/64 -lpcre"

Thanks for your insight re: the missing tools/missing (or whatever it 
is looking for!)

Michael

Hi Michael,

I experienced the same thing in the past.  After you get the R-3.2.2.tar.gz
and uncompress it, enter the R-3.2.2 directory and run the ./configure and
make from there, rather than a different one.

One other thing, you probably already did this, but just in case you might
want to double-check this section as well:

https://cran.r-project.org/doc/manuals/r-release/R-admin.html#AIX

Hope it helps,
~p

-----Original Message-----
From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of aixtools
Sent: Monday, November 23, 2015 4:31 PM
To: r-devel at r-project.org
Subject: Re: [Rd] compile question

Dear all,

Starting all over again with R-devel of 22-11-2015.

After getting the dependancies (minimal graphics initially) I have 
been able to come this far - with a 64-bit build on AIX.



This is my configure call:

#!/usr/bin/ksh
# my_config_test

. /data/prj/cran/64/setup.env

./configure --enable-maintainer-mode  --enable-R-shlib --enable-lto 
--with-libpth-prefix=prefix=/opt \

--prefix=/opt \

        --disable-rpath --with-readline=no --with-x=no \
        --with-cairo=no --with-libpng=no --with-jpeglib=no 
--with-libtiff=no \
        --with-aix-soname=aix

So, modified the configure call, just incase, to be a bit more 'normal', but
still getting a failure with:

creating src/scripts/R.fe
make[3]: Entering directory '/data/prj/cran/64/R-devel/src/scripts'
mkdir -p -- ../../bin
make[3]: Leaving directory '/data/prj/cran/64/R-devel/src/scripts'
make[2]: Leaving directory '/data/prj/cran/64/R-devel/src/scripts'
make[2]: Entering directory '/data/prj/cran/64/R-devel/src/include'
mkdir -p -- ../../include
/bin/sh: ../../tools/missing:  not found.
Makefile:66: recipe for target 'stamp-h.in' failed
make[2]: *** [stamp-h.in] Error 127
make[2]: Leaving directory '/data/prj/cran/64/R-devel/src/include'
Makefile:28: recipe for target 'R' failed
make[1]: *** [R] Error 1
make[1]: Leaving directory '/data/prj/cran/64/R-devel/src'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1

Everything seems top be in place, but I do not know GNU autotools well
enough (I thought that is the origin of 'missing')

root at x072:[/data/prj/cran/64/R-devel]cd src/include
root at x072:[/data/prj/cran/64/R-devel/src/include]find ../.. -name missing
../../tools/missing root at x072:[/data/prj/cran/64/R-devel/src/include]find
../.. -name missing -ls
101915983   11 -rwxrwxr-x  1 2300     2300        11135 Nov 14  2013 
../../tools/missing
root at x072:[/data/prj/cran/64/R-devel/src/include]head  ../../tools/missing
#! /bin/sh # Common stub for a few missing GNU programs while installing.

scriptversion=2006-05-10.23

# Copyright (C) 1996, 1997, 1999, 2000, 2002, 2003, 2004, 2005, 2006
#   Free Software Foundation, Inc.
# Originally by Fran,cois Pinard <pinard at iro.umontreal.ca>, 1996.

# This program is free software; you can redistribute it and/or modify

And the environment is:
root at x072:[/data/prj/cran/64/R-devel]cat ../setup* export 
OBJECT_MODE=64 export CC="gcc -maix64"
export CXX="g++ -maix64"
export F77="gfortran -maix64"
export FC="gfortran -maix64"
export LDFLAGS="-L/opt/lib/64 -lpcre"

Thanks for your insight re: the missing tools/missing (or whatever it 
is looking for!)

Michael

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

On 23 Nov 2015, at 22:30 , aixtools <aixtools at gmail.com> wrote:

./configure --enable-maintainer-mode ...

Two things here

- possibly irrelevant, but I'd avoid building in the source directory. (mkdir ../BUILD ; cd ../BUILD; ../R/configure)

- don't turn on mantainer mode. You are not a maintainer, and if you want to play at being one, I think you need extra tools.

Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com

On 23 Nov 2015, at 22:30 , aixtools<aixtools at gmail.com>  wrote:

./configure --enable-maintainer-mode ...

Two things here

- possibly irrelevant, but I'd avoid building in the source directory. (mkdir ../BUILD ; cd ../BUILD; ../R/configure)

I'll try this. It is a requirement for building gcc.

- don't turn on mantainer mode. You are not a maintainer, and if you want to play at being one, I think you need extra tools.

Nope - do not want to be a maintainer (of R) - just hope to have a easy 
to install package. I was trying the flag because it helped spot things 
while packaging httpd. Would not have it on for normal packaging. I was 
thinking more verbose messages.

Thanks.

Hi Michael,

I experienced the same thing in the past.  After you get the R-3.2.2.tar.gz
and uncompress it, enter the R-3.2.2 directory and run the ./configure and
make from there, rather than a different one.

I have been able to get a 32-bit build, easily, with R-3.1.3, and I 
think also with R-3.2.2. But the 64-bit builds have not been 
straightforward. Unusual and/or unexpected messages.
As a lot of packaging changes have been made - noteably the removal of 
several libraries that must now come from the system - make me think 
that focusing on the R-devel branch is more relevant.

However, I should probably revisit R-3.2.2!

One other thing, you probably already did this, but just in case you might
want to double-check this section as well:

https://cran.r-project.org/doc/manuals/r-release/R-admin.html#AIX

Yes, I have been using this - and it helps.

Hope it helps,

Thanks!

~p

-----Original Message-----
From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of aixtools
Sent: Monday, November 23, 2015 4:31 PM
To: r-devel at r-project.org
Subject: Re: [Rd] compile question

On 2015-11-23 18:52, aixtools wrote:

Dear all,

Starting all over again with R-devel of 22-11-2015.

After getting the dependancies (minimal graphics initially) I have
been able to come this far - with a 64-bit build on AIX.

snip...

On 23 Nov 2015, at 22:30 , aixtools<aixtools at gmail.com>  wrote:

./configure --enable-maintainer-mode ...

Two things here

- possibly irrelevant, but I'd avoid building in the source directory. (mkdir ../BUILD ; cd ../BUILD; ../R/configure)

- don't turn on mantainer mode. You are not a maintainer, and if you want to play at being one, I think you need extra tools.

Took advice, and moving forward. Currently both 32-bit and 64-bit are 
having similar issues - which I prefer to one working perfectly and the 
other having issues.

Main issues are dublicate issues and a missing function - that I would 
expect to be in libm (logf()).

So, for both 32-bit and 64-bit, even with -lm added to LDFLAGS it is 
ending with:

byte-compiling package 'grDevices'
Warning in solve.default(rgb) :
   unable to load shared object 
'/data/prj/cran/32/R-build/modules//lapack.so':
   rtld: 0712-001 Symbol logf was referenced
       from module /data/prj/cran/32/R-build/lib/libRlapack.so(), but a 
runtime definition
             of the symbol was not found.
Error in solve.default(rgb) : LAPACK routines cannot be loaded
Error: unable to load R code in package 'grDevices'
Execution halted
/data/prj/cran/R-devel/share/make/lazycomp.mk:7: recipe for target 
'../../../library/grDevices/R/grDevices.rdb' failed

For the interested, I have captured the screen outputs of the configure 
and make - sub-directory 003 is the last 64-bit I tried, and 
sub-directory 004 is the 32-bit build. See 
http://download.aixtools.net/test/R/2015-11-23/ as I do not want to bog 
down mailers with included files.

* Note that the 32-bit build failed because the 64-bit curl include 
files were still installed. MAYBE it is worth considering having curl
included again in the source distribution. That way the correct include 
files can be included in R directories for packages et al that are being 
built around/based upon R environment.

On 23 Nov 2015, at 22:30 , aixtools<aixtools at gmail.com>  wrote:

./configure --enable-maintainer-mode ...

Two things here

- possibly irrelevant, but I'd avoid building in the source directory. (mkdir ../BUILD ; cd ../BUILD; ../R/configure)

- don't turn on mantainer mode. You are not a maintainer, and if you want to play at being one, I think you need extra tools.

I think I may need the extra tools. Besides autoconf and automake (plus 
gnu m4) - what other tools are needed?

I am not able to generate a good configure (yet).

root at x072:[/data/prj/cran/R-devel]autoconf
configure.ac:254: error: possibly undefined macro: AM_CONDITIONAL
       If this token and others are legitimate, please use m4_pattern_allow.
       See the Autoconf documentation.
configure.ac:655: error: possibly undefined macro: AC_DISABLE_STATIC
configure.ac:727: error: possibly undefined macro: AC_CHECK_LIBM
configure.ac:1982: error: possibly undefined macro: AM_LANGINFO_CODESET
configure.ac:2639: error: possibly undefined macro: AM_NLS
configure.ac:2643: error: possibly undefined macro: AM_GNU_GETTEXT_VERSION
configure.ac:2644: error: possibly undefined macro: AM_GNU_GETTEXT

What I changed...

root at x072:[/data/prj/cran]diff -u R-devel_2015-11-26/configure.ac 
R-devel/configure.ac
--- R-devel_2015-11-26/configure.ac     2015-10-18 16:02:55.000000000 +0000
+++ R-devel/configure.ac        2015-11-26 13:29:28.000000000 +0000
@@ -1300,12 +1300,14 @@
       ##ADD: A symbol of memcpy,memset is exported in libR by expall.
       ##ADD: However, for example, symbol in libc of memcpy is 
__memmove,__memmove64.
       ##ADD: This black magic puts lc before lR and pockets this.
+     ## MAMF: This black magic is no longer working, it seems - getting 
duplicate symbol errors now.
       if test "x${OBJECT_MODE}" = "x64"; then
-       main_ldflags="${wl}-brtl ${wl}-bexpall ${wl}-bpT:0x100000000 
${wl}-bpD:0x110000000 -lc"
+       main_ldflags="${wl}-brtl ${wl}-bexpall ${wl}-bpT:0x100000000 
${wl}-bpD:0x110000000"
       else
-       main_ldflags="${wl}-brtl ${wl}-bexpall -lc"
+       main_ldflags="${wl}-brtl ${wl}-bexpall"
       fi
-     shlib_ldflags="${wl}-brtl ${wl}-G ${wl}-bexpall ${wl}-bnoentry -lc"
+     ## MAMF: This black magic is clearly not working, removed from 
main_ldflags to be consistent
+     shlib_ldflags="${wl}-brtl ${wl}-G ${wl}-bexpall ${wl}-bnoentry"
       SHLIB_LIBADD="\$(LIBM)"
       shlib_cxxldflags="${shlib_ldflags}"
       if test "${GCC}" = yes; then

This removes most of the Duplicate symbol messages, although there are 
still several coming from -lR being included
during the build of the shared libraries (.so files).

Another group of Duplicate symbols is coming from the -lm flag being too 
early in the command syntax (autoconf library discovery, so more 
difficult to "move", or "remove".

And I expect the logf() failure is from a program that was dynamically 
linked but the symbol was ignored (as unref/unknown) for some reason. 
I'll continue looking for that.

Regards,
Michael

p.s. is libR still exporting a libc defined symbol? Or is this just very 
old text?

If you do want to play with maintainer-mode, first check out

https://developer.r-project.org/R-build-prerelease

it is what is done in the nightly builds on OSX. (Actually Mavericks. The comment is old.)

-pd

On 2015-11-23 23:43, peter dalgaard wrote:

On 23 Nov 2015, at 22:30 , aixtools<aixtools at gmail.com>  wrote:

./configure --enable-maintainer-mode ...

Two things here

- possibly irrelevant, but I'd avoid building in the source directory. (mkdir ../BUILD ; cd ../BUILD; ../R/configure)

- don't turn on mantainer mode. You are not a maintainer, and if you want to play at being one, I think you need extra tools.

I think I may need the extra tools. Besides autoconf and automake (plus gnu m4) - what other tools are needed?

I am not able to generate a good configure (yet).

root at x072:[/data/prj/cran/R-devel]autoconf
configure.ac:254: error: possibly undefined macro: AM_CONDITIONAL
     If this token and others are legitimate, please use m4_pattern_allow.
     See the Autoconf documentation.
configure.ac:655: error: possibly undefined macro: AC_DISABLE_STATIC
configure.ac:727: error: possibly undefined macro: AC_CHECK_LIBM
configure.ac:1982: error: possibly undefined macro: AM_LANGINFO_CODESET
configure.ac:2639: error: possibly undefined macro: AM_NLS
configure.ac:2643: error: possibly undefined macro: AM_GNU_GETTEXT_VERSION
configure.ac:2644: error: possibly undefined macro: AM_GNU_GETTEXT

What I changed...

root at x072:[/data/prj/cran]diff -u R-devel_2015-11-26/configure.ac R-devel/configure.ac
--- R-devel_2015-11-26/configure.ac     2015-10-18 16:02:55.000000000 +0000
+++ R-devel/configure.ac        2015-11-26 13:29:28.000000000 +0000
@@ -1300,12 +1300,14 @@
     ##ADD: A symbol of memcpy,memset is exported in libR by expall.
     ##ADD: However, for example, symbol in libc of memcpy is __memmove,__memmove64.
     ##ADD: This black magic puts lc before lR and pockets this.
+     ## MAMF: This black magic is no longer working, it seems - getting duplicate symbol errors now.
     if test "x${OBJECT_MODE}" = "x64"; then
-       main_ldflags="${wl}-brtl ${wl}-bexpall ${wl}-bpT:0x100000000 ${wl}-bpD:0x110000000 -lc"
+       main_ldflags="${wl}-brtl ${wl}-bexpall ${wl}-bpT:0x100000000 ${wl}-bpD:0x110000000"
     else
-       main_ldflags="${wl}-brtl ${wl}-bexpall -lc"
+       main_ldflags="${wl}-brtl ${wl}-bexpall"
     fi
-     shlib_ldflags="${wl}-brtl ${wl}-G ${wl}-bexpall ${wl}-bnoentry -lc"
+     ## MAMF: This black magic is clearly not working, removed from main_ldflags to be consistent
+     shlib_ldflags="${wl}-brtl ${wl}-G ${wl}-bexpall ${wl}-bnoentry"
     SHLIB_LIBADD="\$(LIBM)"
     shlib_cxxldflags="${shlib_ldflags}"
     if test "${GCC}" = yes; then

This removes most of the Duplicate symbol messages, although there are still several coming from -lR being included
during the build of the shared libraries (.so files).

Another group of Duplicate symbols is coming from the -lm flag being too early in the command syntax (autoconf library discovery, so more difficult to "move", or "remove".

And I expect the logf() failure is from a program that was dynamically linked but the symbol was ignored (as unref/unknown) for some reason. I'll continue looking for that.

Regards,
Michael

p.s. is libR still exporting a libc defined symbol? Or is this just very old text?

Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com

If you do want to play with maintainer-mode, first check out

https://developer.r-project.org/R-build-prerelease

it is what is done in the nightly builds on OSX. (Actually Mavericks. The comment is old.)

-pd

Very helpful.
I only ran the "sync", aclocal and autoconf commands - so what else 
(extra) does 'maintainer-mode' do? On AIX I could think of adding 
-Wl,-b:map=${object}.map (one of my new tricks for finding which library 
is going to be used).

Other comments/questions:

a) the small change in configure.ac got rid of a lot of error/rather 
warning messages from the binder (ld: 0711-224 WARNING: Duplicate 
symbol:.myFavSymbol)
b) lost of ld warnings are going to be more difficult to resolve - as 
they are being generated by the autoconf processing of gfortran -v - and 
the placement of XXX_LIBADD flags (basically, in the way now - still 
looking at where they get placed - are there "switches" for Makefile.in 
processing?
c) looking backward and forward - a hint on where to look is 
appreciated. In one of the m4 files /usr/local/include is defined as the 
default include directory. However, it does not actually use 
${prefix}/include which is what I would want. the looking backward part 
- a month ago when I was working with R to get it ready for a project 
(and then 32-bit was too small a memory model) - while compiling and 
installing additional packages I was forced to use /usr/local/lib and 
/usr/local/include. Would like to get away from that, if possible.
d) the biggest bother of all ;-| - a bug in AIX, and an undefined symbol 
in /usr/lib/libm.a - You can look for a LONG time, and never get further 
when an error like that sneaks in with an update!

On 2015-11-26 17:15, peter dalgaard wrote:

If you do want to play with maintainer-mode, first check out

https://developer.r-project.org/R-build-prerelease

it is what is done in the nightly builds on OSX. (Actually Mavericks. The comment is old.)

-pd

Very helpful.
I only ran the "sync", aclocal and autoconf commands - so what else (extra) does 'maintainer-mode' do? On AIX I could think of adding -Wl,-b:map=${object}.map (one of my new tricks for finding which library is going to be used).

The easiest way to find out is to look through the Makefile.in-s for lines with @MAINTAINER_MODE_TRUE at . E.g. in  src/include/Makefile.in 

$(srcdir)/config.h.in: @MAINTAINER_MODE_TRUE@ $(srcdir)/stamp-h.in
$(srcdir)/stamp-h.in: @MAINTAINER_MODE_TRUE@ $(top_srcdir)/configure.ac $(top_builddir)/aclocal.m4
        @TBD=`cd $(top_builddir); pwd`; \
          cd $(top_srcdir) && $(AUTOHEADER) -I $${TBD}
        @$(ECHO) timestamp > $@ 2> /dev/null

(which I think is what bit you via a missing "autoheader" command). It usually boils down to disabling certain dependencies.)

Notice that in maintainer-mode, the build _will_ (sometimes) modify files under $(top_srcdir), even if building in a separate directory. That is otherwise a big no-no, which is essentially why it is reserved for maintainers. (And the nightly builds try to ensure that not all even maintainers will be modifying certain source files. The tools for making them are somewhat flaky, and we don't all have the appropriate versions.)

Other comments/questions:

a) the small change in configure.ac got rid of a lot of error/rather warning messages from the binder (ld: 0711-224 WARNING: Duplicate symbol:.myFavSymbol)
b) lost of ld warnings are going to be more difficult to resolve - as they are being generated by the autoconf processing of gfortran -v - and the placement of XXX_LIBADD flags (basically, in the way now - still looking at where they get placed - are there "switches" for Makefile.in processing?
c) looking backward and forward - a hint on where to look is appreciated. In one of the m4 files /usr/local/include is defined as the default include directory. However, it does not actually use ${prefix}/include which is what I would want. the looking backward part - a month ago when I was working with R to get it ready for a project (and then 32-bit was too small a memory model) - while compiling and installing additional packages I was forced to use /usr/local/lib and /usr/local/include. Would like to get away from that, if possible.
d) the biggest bother of all ;-| - a bug in AIX, and an undefined symbol in /usr/lib/libm.a - You can look for a LONG time, and never get further when an error like that sneaks in with an update!

Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com

The easiest way to find out is to look through the Makefile.in-s for lines with @MAINTAINER_MODE_TRUE at . E.g. in  src/include/Makefile.in

$(srcdir)/config.h.in: @MAINTAINER_MODE_TRUE@ $(srcdir)/stamp-h.in
$(srcdir)/stamp-h.in: @MAINTAINER_MODE_TRUE@ $(top_srcdir)/configure.ac $(top_builddir)/aclocal.m4
         @TBD=`cd $(top_builddir); pwd`; \
           cd $(top_srcdir)&&  $(AUTOHEADER) -I $${TBD}
         @$(ECHO) timestamp>  $@ 2>  /dev/null

(which I think is what bit you via a missing "autoheader" command). It usually boils down to disabling certain dependencies.)

Notice that in maintainer-mode, the build_will_  (sometimes) modify files under $(top_srcdir), even if building in a separate directory. That is otherwise a big no-no, which is essentially why it is reserved for maintainers. (And the nightly builds try to ensure that not all even maintainers will be modifying certain source files. The tools for making them are somewhat flaky, and we don't all have the appropriate versions.)

For now, I shall just be a humble-packager.
When I have something I would like to suggest as a change, specific for 
AIX, to configure.ac how can I submit that.

I shall hold back on changes to various Makefile.in (still need to learn 
how automake could help with platform specific things. I see comments of 
platform specific things (e.g., for Solaris), but not how they get 
activated/used.

Regards,
Michael