Skip to content

Latest R-devel build failing on OS X

21 messages · Keith O'Hara, frederik at ofb.net, Martin Maechler +1 more

#
Dear R-devel,

The latest version of R-devel (05-17) is throwing an error for me when building on OS X (v 10.11.4):

making Rembedded.d from Rembedded.c
making dynload.d from dynload.c
making system.d from system.c
making sys-unix.d from sys-unix.c
making sys-std.d from sys-std.c
making X11.d from X11.c
clang -I. -I../../src/include -I../../src/include  -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -Wall -mtune=core2 -g -O2  -c Rembedded.c -o Rembedded.o
clang -I. -I../../src/include -I../../src/include  -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -Wall -mtune=core2 -g -O2  -c dynload.c -o dynload.o
clang -I. -I../../src/include -I../../src/include  -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -Wall -mtune=core2 -g -O2  -c system.c -o system.o
clang -I. -I../../src/include -I../../src/include  -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -Wall -mtune=core2 -g -O2  -c sys-unix.c -o sys-unix.o
clang -I. -I../../src/include -I../../src/include  -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -Wall -mtune=core2 -g -O2  -c sys-std.c -o sys-std.o
sys-std.c:592:5: warning: implicit declaration of function 'RL_UNSETSTATE' is invalid in C99
     [-Wimplicit-function-declaration]
   RL_UNSETSTATE(RL_STATE_ISEARCH | RL_STATE_NSEARCH | RL_STATE_VIMOTION |
   ^
sys-std.c:592:19: error: use of undeclared identifier 'RL_STATE_ISEARCH'
   RL_UNSETSTATE(RL_STATE_ISEARCH | RL_STATE_NSEARCH | RL_STATE_VIMOTION |
                 ^
sys-std.c:592:38: error: use of undeclared identifier 'RL_STATE_NSEARCH'
   RL_UNSETSTATE(RL_STATE_ISEARCH | RL_STATE_NSEARCH | RL_STATE_VIMOTION |
                                    ^
sys-std.c:592:57: error: use of undeclared identifier 'RL_STATE_VIMOTION'
   RL_UNSETSTATE(RL_STATE_ISEARCH | RL_STATE_NSEARCH | RL_STATE_VIMOTION |
                                                       ^
sys-std.c:593:5: error: use of undeclared identifier 'RL_STATE_NUMERICARG'
                 RL_STATE_NUMERICARG | RL_STATE_MULTIKEY);
                 ^
sys-std.c:593:27: error: use of undeclared identifier 'RL_STATE_MULTIKEY'
                 RL_STATE_NUMERICARG | RL_STATE_MULTIKEY);
                                       ^
sys-std.c:596:40: error: use of undeclared identifier 'rl_mark'
   rl_line_buffer[rl_point = rl_end = rl_mark = 0] = 0;
                                      ^
sys-std.c:597:5: error: use of undeclared identifier 'rl_done'
   rl_done = 1;
   ^
sys-std.c:998:7: warning: implicit declaration of function 'rl_resize_terminal' is invalid in C99
     [-Wimplicit-function-declaration]
                   rl_resize_terminal();
                   ^
2 warnings and 7 errors generated.
make[3]: *** [sys-std.o] Error 1
make[2]: *** [R] Error 2
make[1]: *** [R] Error 1
make: *** [R] Error 1




My configuration information:

R is now configured for x86_64-apple-darwin15.4.0

 Source directory:          .
 Installation directory:    /Builds/R-devel

 C compiler:                clang  -Wall -mtune=core2 -g -O2
 Fortran 77 compiler:       gfortran-4.8  -g -O2

 C++ compiler:              clang++  -Wall -mtune=core2 -g -O2
 C++11 compiler:            clang++  -std=c++11 -Wall -mtune=core2 -g -O2
 Fortran 90/95 compiler:    gfortran-4.8 -Wall -g -O2
 Obj-C compiler:	     clang -Wall -mtune=core2 -g -O2 -fobjc-exceptions

 Interfaces supported:      aqua, tcltk
 External libraries:        readline, BLAS(OpenBLAS), LAPACK(in blas), curl
 Additional capabilities:   PNG, JPEG, TIFF, NLS, cairo, ICU
 Options enabled:           shared R library, R profiling, memory profiling

 Capabilities skipped:      
 Options not enabled:       shared BLAS

 Recommended packages:      yes


Apologies in advance if I have incorrectly formatted the issue or omitted something important.

Kind regards,
Keith
#
Yes, the nightly build is broken in a similar, but different way. See below.

Both seem to be readline related, so Frederick Eaton's patches, which Martin committed yesterday are the likely culprit. I had  actually tested them and things seemed to work, but it was on a different machine and not a completely clean build.

-pd

.....
ranlib: file: libR.a(printf.o) has no symbols
gcc   -L/usr/X11R6/lib -o R.bin Rmain.o CommandLineArgs.o Rdynload.o Renviron.o RNG.o agrep.o apply.o arithmetic.o array.o attrib.o bind.o builtin.o character.o coerce.o colors.o complex.o connections.o context.o cum.o dcf.o datetime.o debug.o deparse.o devices.o dotcode.o dounzip.o dstruct.o duplicate.o edit.o engine.o envir.o errors.o eval.o format.o gevents.o gram.o gram-ex.o graphics.o grep.o identical.o inlined.o inspect.o internet.o iosupport.o lapack.o list.o localecharset.o logic.o main.o mapply.o match.o memory.o names.o objects.o options.o paste.o platform.o plot.o plot3d.o plotmath.o print.o printarray.o printvector.o printutils.o qsort.o radixsort.o random.o raw.o registration.o relop.o rlocale.o saveload.o scan.o seq.o serialize.o sort.o source.o split.o sprintf.o startup.o subassign.o subscript.o subset.o summary.o sysutils.o times.o unique.o util.o version.o g_alab_her.o g_cntrlify.o g_fontdb.o g_her_glyph.o xxxpr.o   `ls ../unix/*.o ../appl/*.o ../nmath/*.o` ../extra/
tre/libtre.a  ../extra/intl/libintl.a ../extra/tzone/libtz.a -L../../lib/x86_64 -lRblas -L/usr/lib/gcc/i686-apple-darwin11/4.2.1/x86_64 -L/usr/lib/gcc/i686-apple-darwin11/4.2.1 -L/usr/lib -lgfortran    -Wl,-framework -Wl,CoreFoundation -lreadline  -lpcre -llzma -lbz2 -lz -licucore -lm -llzma -liconv
Undefined symbols for architecture x86_64:
 "_rl_mark", referenced from:
     _popReadline in sys-std.o
 "_rl_readline_state", referenced from:
     _popReadline in sys-std.o
 "_rl_resize_terminal", referenced from:
     _Rstd_ReadConsole in sys-std.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [R.bin] Error 1
On 18 May 2016, at 14:18 , Keith O'Hara <keith.ohara at nyu.edu> wrote:

            

  
    
#
Indeed a problem.  I'm pretty sure that RL_UNSETSTATE() exists
in newer versions of readline  but not in older ones (and hence is
declared in newer versions of readline.h, but not in older
ones).

It seems people still do have older versions of readline.h 
... and it may be interesting why some versions of OSX (Peter's)
has a new readline and some (Keith') don't.

I'm committing an experimental fix using  #if ... s and the
readline version number.
It does keep fixing the bug on my platform (Fedora 22 Linux,
readline 6.3)

and it may help in Keith' setup.  Please do check it, Keith.
(svn rev >= 70631)

Martin

            
#
Thanks for the fix, Martin.

The build runs smoothly now, with just a warning remaining:

sys-std.c:1002:7: warning: implicit declaration of function 'rl_resize_terminal' is invalid in C99
      [-Wimplicit-function-declaration]
                    rl_resize_terminal();
                    ^
1 warning generated.

Best,
Keith
#
Er, Peter has two machines, one an ancient iMac still running Mavericks, the other a nearly as old MB Air running Yosemite. The MBAir apparently worked, the iMac not. *Both* of them has readline 5.2 in /usr/local and 6.2/3 in /opt/local, but the MBAir binary has

Notice incidentally that 5.2 is what Simon currently supports on R.research.att.com.

Peter-Dalgaards-MacBook-Air:BUILD pd$ otool -L bin/exec/x86_64/R 
bin/exec/x86_64/R:
....
	/usr/local/lib/libreadline.5.2.dylib (compatibility version 5.0.0, current version 5.2.0)

With the recent update, the iMac still fails with

gcc   -L/usr/X11R6/lib -o R.bin Rmain.o CommandLineArgs.o Rdynload.o Renviron.o RNG.o agrep.o apply.o arithmetic.o array.o attrib.o bind.o builtin.o character.o coerce.o colors.o complex.o connections.o context.o cum.o dcf.o datetime.o debug.o deparse.o devices.o dotcode.o dounzip.o dstruct.o duplicate.o edit.o engine.o envir.o errors.o eval.o format.o gevents.o gram.o gram-ex.o graphics.o grep.o identical.o inlined.o inspect.o internet.o iosupport.o lapack.o list.o localecharset.o logic.o main.o mapply.o match.o memory.o names.o objects.o options.o paste.o platform.o plot.o plot3d.o plotmath.o print.o printarray.o printvector.o printutils.o qsort.o radixsort.o random.o raw.o registration.o relop.o rlocale.o saveload.o scan.o seq.o serialize.o sort.o source.o split.o sprintf.o startup.o subassign.o subscript.o subset.o summary.o sysutils.o times.o unique.o util.o version.o g_alab_her.o g_cntrlify.o g_fontdb.o g_her_glyph.o xxxpr.o   `ls ../unix/*.o ../appl/*.o ../nmath/*.o` ../extra/tre/libtre.a  ../extra/intl/libintl.a ../extra/tzone/libtz.a -L../../lib/x86_64 -lRblas -L/usr/lib/gcc/i686-apple-darwin11/4.2.1/x86_64 -L/usr/lib/gcc/i686-apple-darwin11/4.2.1 -L/usr/lib -lgfortran    -Wl,-framework -Wl,CoreFoundation -lreadline  -lpcre -llzma -lbz2 -lz -licucore -lm -llzma -liconv
Undefined symbols for architecture x86_64:
  "_rl_resize_terminal", referenced from:
      _Rstd_ReadConsole in sys-std.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [R.bin] Error 1
make[2]: *** [R] Error 2
make[1]: *** [R] Error 1
make: *** [R] Error 1

and the MBAir still builds... 

Confused!,

-pd

  
    
#
Ah, got it. For some ancient reason config.site on that machine had 

LDFLAGS=-L/usr/X11R6/lib

in config.site, and that prevented configure from inserting -L /usr/local/lib, so linked /usr/lib/libreadline.dylib, which is the Apple-supplied one, which possibly is really libedit...

-p

  
    
#
Spoke too soon, both systems now build, but neither has the original bugs fixed....

(Incidentally, I realized why the ctl-R...ctl-C bug never bit me: The emacs habit is to exit isearch with ctl-G and that works flawlessly.) 

-pd

  
    
#
Readline <= 6.2 shouldn't require the SIGWINCH patch, so if older
versions have trouble finding rl_resize_terminal then you could wrap a
macro around that part.

The isearch C-c bug has existed forever, according to Chet Ramey. Yes,
I had to retrain myself to use C-g to exit isearch but it's confusing.
It would be nice to fix this C-c bug on older versions too, but my
solution used some global variables and I'm not sure which Readline
version they date from.

Out of curiosity, did you delete the HAVE_READLINE_READLINE_H block?

Thanks so much for working this out!

Frederick

P.S. I do use Emacs too, but I have some bindings which stuff the
current region into a numbered 'GNU screen' window (where R is
running) - I was never a fan of inferior Emacs processes. Also a
keybinding which stuffs "source('CURRENT_FILE')\n". And a rxvt-unicode
'url-select' extension which copies text after a "> " prompt, to get
commands I edited back into Emacs. That's why I still use the basic
interface.
On Wed, May 18, 2016 at 10:58:59PM +0200, peter dalgaard wrote:
#
> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    > if older versions have trouble finding rl_resize_terminal
    > then you could wrap a macro around that part.

I find python related patches that use

#ifdef HAVE_RL_RESIZE_TERMINAL

so they must have configured for that.  We could and probably
should do the same,  but as a Linux_only guy currently (even
basically only one flavor of Linux), I'd appreciate others to
produce code for that.

    > The isearch C-c bug has existed forever, according to Chet Ramey.

I see.  As your patch seems to only work for readline (>=) 6.3,
we have solved part of the problems.
If those who use olders readlines are willing to test and
provide (well looking, as yours) patches for earlier versions,
they should be welcome too.



    > Yes, I had to retrain myself to use C-g to exit
    > isearch but it's confusing.  It would be nice to fix this
    > C-c bug on older versions too, but my solution used some
    > global variables and I'm not sure which Readline version
    > they date from.

    > Out of curiosity, did you delete the
    > HAVE_READLINE_READLINE_H block?

Not yet...I've even kept your suggestion about it in the
source comments.
Note you can always point your web browser at
  https://svn.r-project.org/trunk/R/  which contains upto the
minute current R development sources (it *is* our apache based
subversion server), i.e., the file in this case is
  https://svn.r-project.org/trunk/R/src/unix/sys-std.c

{or you can use the github *mirror* of the svn server which provides
 version/revision/log etc  ((this be available for svn as well,
 but we (I) had chosen not to provide (via apache modules) just
 to keep our server as bare bones as possible and  hence less
 vulnerable to viciousities))
}
 
    > Thanks so much for working this out!
    > Frederick

    > P.S. I do use Emacs too, but I have some bindings which
    > stuff the current region into a numbered 'GNU screen'
    > window (where R is running) - I was never a fan of
    > inferior Emacs processes. Also a keybinding which stuffs
    > "source('CURRENT_FILE')\n". And a rxvt-unicode
    > 'url-select' extension which copies text after a "> "
    > prompt, to get commands I edited back into Emacs. That's
    > why I still use the basic interface.

    > On Wed, May 18, 2016 at 10:58:59PM +0200, peter dalgaard
> wrote:
>> Spoke too soon, both systems now build, but neither has
    >> the original bugs fixed....
    >> 
    >> (Incidentally, I realized why the ctl-R...ctl-C bug never
    >> bit me: The emacs habit is to exit isearch with ctl-G and
    >> that works flawlessly.)
    >> 
    >> -pd
    >> 
    >> > On 18 May 2016, at 22:40 , peter dalgaard
>> <pdalgd at gmail.com> wrote:
>> > 
    >> > Ah, got it. For some ancient reason config.site on that
    >> machine had
    >> > 
    >> > LDFLAGS=-L/usr/X11R6/lib
    >> > 
    >> > in config.site, and that prevented configure from
    >> inserting -L /usr/local/lib, so linked
    >> /usr/lib/libreadline.dylib, which is the Apple-supplied
    >> one, which possibly is really libedit...
    >> > 
    >> > -p
    >> > 
    >> > 
    >> >> On 18 May 2016, at 22:01 , peter dalgaard
>> <pdalgd at gmail.com> wrote:
>> >> 
    >> >> gcc -L/usr/X11R6/lib -o R.bin Rmain.o
    >> CommandLineArgs.o Rdynload.o Renviron.o RNG.o agrep.o
    >> apply.o arithmetic.o array.o attrib.o bind.o builtin.o
    >> character.o coerce.o colors.o complex.o connections.o
    >> context.o cum.o dcf.o datetime.o debug.o deparse.o
    >> devices.o dotcode.o dounzip.o dstruct.o duplicate.o
    >> edit.o engine.o envir.o errors.o eval.o format.o
    >> gevents.o gram.o gram-ex.o graphics.o grep.o identical.o
    >> inlined.o inspect.o internet.o iosupport.o lapack.o
    >> list.o localecharset.o logic.o main.o mapply.o match.o
    >> memory.o names.o objects.o options.o paste.o platform.o
    >> plot.o plot3d.o plotmath.o print.o printarray.o
    >> printvector.o printutils.o qsort.o radixsort.o random.o
    >> raw.o registration.o relop.o rlocale.o saveload.o scan.o
    >> seq.o serialize.o sort.o source.o split.o sprintf.o
    >> startup.o subassign.o subscript.o subset.o summary.o
    >> sysutils.o times.o unique.o util.o version.o g_alab_her.o
    >> g_cntrlify.o g_fontdb.o g_her_glyph.o xxxpr.o `ls
    >> ../unix/*.o ../appl/*.o ../nmath/*.o`
    >> ../extra/tre/libtre.a ../extra/intl/libintl.a
    >> ../extra/tzone/libtz.a -L../../lib/x86_64 -lRblas
    >> -L/usr/lib/gcc/i686-apple-darwin11/4.2.1/x86_64
    >> -L/usr/lib/gcc/i686-apple-darwin11/4.2.1 -L/usr/lib
    >> -lgfortran -Wl,-framework -Wl,CoreFoundation -lreadline
    >> -lpcre -llzma -lbz2 -lz -licucore -lm -llzma -liconv >>
    >> Undefined symbols for architecture x86_64: >>
    >> "_rl_resize_terminal", referenced from: >>
    >> _Rstd_ReadConsole in sys-std.o >> ld: symbol(s) not found
    >> for architecture x86_64 >> clang: error: linker command
    >> failed with exit code 1 (use -v to see invocation) >>
    >> make[3]: *** [R.bin] Error 1 >> make[2]: *** [R] Error 2
    >> >> make[1]: *** [R] Error 1 >> make: *** [R] Error 1
    >> >> 
    >> >> and the MBAir still builds...
    >> >> 
    >> >> Confused!,
    >> >> 
    >> >> -pd
    >> > 
    >> > -- 
    >> > Peter Dalgaard, Professor, > Center for Statistics,
    >> Copenhagen Business School > Solbjerg Plads 3, 2000
    >> Frederiksberg, Denmark > Phone: (+45)38153501 > Office: A
    >> 4.23 > Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
    >> > 
    >> > 
    >> > 
    >> > 
    >> > 
    >> > 
    >> > 
    >> > 
    >> > 
    >> 
    >> -- 
    >> Peter Dalgaard, Professor, Center for Statistics,
    >> Copenhagen Business School Solbjerg Plads 3, 2000
    >> Frederiksberg, Denmark Phone: (+45)38153501 Office: A
    >> 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
    >> 
    >> 
    >> 
    >> 
    >> 
    >> 
    >> 
    >> 
    >>
#
>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    >> if older versions have trouble finding rl_resize_terminal
    >> then you could wrap a macro around that part.

    > I find python related patches that use

    > #ifdef HAVE_RL_RESIZE_TERMINAL

    > so they must have configured for that.  We could and
    > probably should do the same, but as a Linux_only guy
    > currently (even basically only one flavor of Linux), I'd
    > appreciate others to produce code for that.

Actually that was easy (in hindsight.. I took too long!)
enough, so I've now committed

------------------------------------------------------------------------
r70632 | maechler | 2016-05-19 10:59:51 +0200 (Thu, 19 May 2016) | 1 line
Changed paths:
   M configure
   M configure.ac
   M src/include/config.h.in
   M src/unix/sys-std.c

check for rl_resize_terminal() now
------------------------------------------------------------------------

... and Keith should even not see the warning anymore
(nor Peter the error, when compiling using readline 5.x instead of 6.[23]).



    >> The isearch C-c bug has existed forever, according to
    >> Chet Ramey.

    > I see.  As your patch seems to only work for readline (>=)
    > 6.3, we have solved part of the problems.  If those who
    > use olders readlines are willing to test and provide (well
    > looking, as yours) patches for earlier versions, they
    > should be welcome too.



    >> Yes, I had to retrain myself to use C-g to exit isearch
    >> but it's confusing.  It would be nice to fix this C-c bug
    >> on older versions too, but my solution used some global
    >> variables and I'm not sure which Readline version they
    >> date from.

    >> Out of curiosity, did you delete the
    >> HAVE_READLINE_READLINE_H block?

    > Not yet...I've even kept your suggestion about it in the
    > source comments.  Note you can always point your web
    > browser at https://svn.r-project.org/trunk/R/ which
    > contains upto the minute current R development sources (it
    > *is* our apache based subversion server), i.e., the file
    > in this case is
    > https://svn.r-project.org/trunk/R/src/unix/sys-std.c

    > {or you can use the github *mirror* of the svn server
    > which provides version/revision/log etc ((this be
    > available for svn as well, but we (I) had chosen not to
    > provide (via apache modules) just to keep our server as
    > bare bones as possible and hence less vulnerable to
    > viciousities)) }
5 days later
#
Can you (Frederick, Peter, Keith, but ideally others, too)
confirm that you don't see any problems anymore, when building a
version of R-devel from sources that are newer 
than (or equal to)  svn revision 70632 (2016-05-19 10:59:51, see below)?

I'm asking because the question is open if these should be
"back ported" to R 3.3.0 patched or not.

Best regards,
Martin
>>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    >>> if older versions have trouble finding rl_resize_terminal
    >>> then you could wrap a macro around that part.

    >> I find python related patches that use

    >> #ifdef HAVE_RL_RESIZE_TERMINAL

    >> so they must have configured for that.  We could and
    >> probably should do the same, but as a Linux_only guy
    >> currently (even basically only one flavor of Linux), I'd
    >> appreciate others to produce code for that.

    > Actually that was easy (in hindsight.. I took too long!)
    > enough, so I've now committed

    > ------------------------------------------------------------------------
    > r70632 | maechler | 2016-05-19 10:59:51 +0200 (Thu, 19 May 2016) | 1 line
    > Changed paths:
    > M configure
    > M configure.ac
    > M src/include/config.h.in
    > M src/unix/sys-std.c

    > check for rl_resize_terminal() now
    > ------------------------------------------------------------------------

    > ... and Keith should even not see the warning anymore
    > (nor Peter the error, when compiling using readline 5.x instead of 6.[23]).


    [...........]
#
I had a regression in config.site so the nightly build didn't. Retrying....

Looks like it will build, but the ctl-R, ctl-C bug is still present on OSX (w/Simon's libs). This _was_ fixed for a while, was it not?

(The NEWS entry is also wrong: The issue existed before readline 6.3)

-pd
On 24 May 2016, at 12:55 , Martin Maechler <maechler at stat.math.ethz.ch> wrote:

            

  
    
#
On my machine (iMac w/ El Capitan (10.11.4)), svn rev. 70662 builds without any errors (and the warning I mentioned before is now gone too).

K
#
> I had a regression in config.site so the nightly build didn't. Retrying....
    > Looks like it will build, but the ctl-R, ctl-C bug is still present on OSX (w/Simon's libs). This _was_ fixed for a while, was it not?

I thought it was never fixed, for readline versions 5.x (or all
of readline_version < 6.3 ?) because the patch assumed features
not available, e.g., for Frederik (who got compilation errors
which I think you confirmed on pre-6 readline).

I remember you having two different readlines installed on OSX
but the standard Mac binary (from CRAN, i.e. Simon) would use
the old readline version ?

so that whole  resetReadline() solution is now conditionalized inside

#if defined(RL_READLINE_VERSION) && RL_READLINE_VERSION >= 0x0603
   ...
   ...
#endif

and hence the previous code (which is buggy) is used for
readline versions < 6.3.
As a consequence the bug is only fixed for readline >= 6.3,
because the current patch did not compile and hence seemed not
appropriate for  readline < 6.3 (and hence the above conditionalization).


    > (The NEWS entry is also wrong: The issue existed before readline 6.3)

Aah.. you are right. The API change with 6.3 was for the other, the
"SIGWINCH" bug.

Here's a an update proposal for that NEWS entry :

         ? The API for readline libraries >= 6.3 had changed such
           terminal window resizes where no longer properly signalled
           (PR#16604).  Also, ?Ctrl C? in incremental search behaved
           confusingly in R (unix) consoles (PR#16603) also for older
           readline versions. These have been fixed (for readline >=
           6.3 only), thanks to patches by Frederick Eaton.


Martin

    > -pd
> On 24 May 2016, at 12:55 , Martin Maechler <maechler at stat.math.ethz.ch> wrote:
>> 
    >> Can you (Frederick, Peter, Keith, but ideally others, too)
    >> confirm that you don't see any problems anymore, when building a
    >> version of R-devel from sources that are newer 
    >> than (or equal to)  svn revision 70632 (2016-05-19 10:59:51, see below)?
    >> 
    >> I'm asking because the question is open if these should be
    >> "back ported" to R 3.3.0 patched or not.
    >> 
    >> Best regards,
    >> Martin
    >> 
    >>>>>>> Martin Maechler <maechler at stat.math.ethz.ch>
    >>>>>>> on Thu, 19 May 2016 11:02:48 +0200 writes:
    >> 
    >>>>>>> <frederik at ofb.net>
    >>>>>>> on Wed, 18 May 2016 15:03:31 -0700 writes:
    >> 
    >>>>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    >>>>> if older versions have trouble finding rl_resize_terminal
    >>>>> then you could wrap a macro around that part.
    >> 
    >>>> I find python related patches that use
    >> 
    >>>> #ifdef HAVE_RL_RESIZE_TERMINAL
    >> 
    >>>> so they must have configured for that.  We could and
    >>>> probably should do the same, but as a Linux_only guy
    >>>> currently (even basically only one flavor of Linux), I'd
    >>>> appreciate others to produce code for that.
    >> 
    >>> Actually that was easy (in hindsight.. I took too long!)
    >>> enough, so I've now committed
    >> 
    >>> ------------------------------------------------------------------------
    >>> r70632 | maechler | 2016-05-19 10:59:51 +0200 (Thu, 19 May 2016) | 1 line
    >>> Changed paths:
    >>> M configure
    >>> M configure.ac
    >>> M src/include/config.h.in
    >>> M src/unix/sys-std.c
    >> 
    >>> check for rl_resize_terminal() now
    >>> ------------------------------------------------------------------------
    >> 
    >>> ... and Keith should even not see the warning anymore
    >>> (nor Peter the error, when compiling using readline 5.x instead of 6.[23]).
    >> 
    >> 
    >> [...........]

    > -- 
    > Peter Dalgaard, Professor,
    > Center for Statistics, Copenhagen Business School
    > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
    > Phone: (+45)38153501
    > Office: A 4.23
    > Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
#
I agree with Martin's summary of the situation, and with the updated
NEWS entry.

I'm not familiar with Subversion, can you tell me the command to use?

(I tried "svn co https://svn.r-project.org/R/" but it seems to be
downloading all branches)

Frederick
On Tue, May 24, 2016 at 04:30:11PM +0200, Martin Maechler wrote:
#
svn checkout https://svn.r-project.org/R/trunk/ <target-directory>
#
Thanks. OK, I get

    $ ./configure --prefix=$HOME/r-svn-test --cache-file=config.cache STRIP=true && make -j8
    ...
    checking for Fortran flag to compile .f95 files... none
    checking for gfortran option to support OpenMP... -fopenmp
    checking for recommended packages... ls: cannot access './src/library/Recommended/MASS_*.tar.gz': No such file or directory
    no
    configure: error: Some of the recommended packages are missing
      Use --without-recommended-packages if this was intentional

However, when I configure --without-recommended-packages and make
install, the build works and both Readline issues are fixed in the
result.

Thank you!
On Tue, May 24, 2016 at 12:47:43PM -0400, Keith O'Hara wrote:
#
> svn checkout https://svn.r-project.org/R/trunk/ <target-directory>

yes, indeed. thank you, Keith.

and from then on only

    cd <target-directory>
    svn up

    (which is short for 'svn update').

Another hint: Then do *not* build in the source directory but in
what we called a "build directory"; i.e., something like
(from scratch; including the only-once needed "checkout") :

     svn checkout https://svn.r-project.org/R/trunk/ R
     cd R
     tools/rsync-recommended
     mkdir ../build-R
     cd ../build-R
     ../R/configure
     make
     make check

and I then never run 'make install', but rather use symbolic
link from
     ..../build-R/bin/R  to something like  ~/bin/R-devel 
i.e.,
     cd ~/bin
     ln -s ..../build-R/bin/R R-devel

Martin
>> On May 24, 2016, at 12:45 PM, frederik at ofb.net wrote:
>> 
    >> I agree with Martin's summary of the situation, and with the updated
    >> NEWS entry.
    >> 
    >> I'm not familiar with Subversion, can you tell me the command to use?
    >> 
    >> (I tried "svn co https://svn.r-project.org/R/" but it seems to be
    >> downloading all branches)
    >> 
    >> Frederick
    >>
>> On Tue, May 24, 2016 at 04:30:11PM +0200, Martin Maechler wrote:
>>>>>>>> peter dalgaard <pdalgd at gmail.com>
    >>>>>>>> on Tue, 24 May 2016 13:47:27 +0200 writes:
    >>> 
    >>>> I had a regression in config.site so the nightly build didn't. Retrying....
    >>>> Looks like it will build, but the ctl-R, ctl-C bug is still present on OSX (w/Simon's libs). This _was_ fixed for a while, was it not?
    >>> 
    >>> I thought it was never fixed, for readline versions 5.x (or all
    >>> of readline_version < 6.3 ?) because the patch assumed features
    >>> not available, e.g., for Frederik (who got compilation errors
    >>> which I think you confirmed on pre-6 readline).
    >>> 
    >>> I remember you having two different readlines installed on OSX
    >>> but the standard Mac binary (from CRAN, i.e. Simon) would use
    >>> the old readline version ?
    >>> 
    >>> so that whole  resetReadline() solution is now conditionalized inside
    >>> 
    >>> #if defined(RL_READLINE_VERSION) && RL_READLINE_VERSION >= 0x0603
    >>> ...
    >>> ...
    >>> #endif
    >>> 
    >>> and hence the previous code (which is buggy) is used for
    >>> readline versions < 6.3.
    >>> As a consequence the bug is only fixed for readline >= 6.3,
    >>> because the current patch did not compile and hence seemed not
    >>> appropriate for  readline < 6.3 (and hence the above conditionalization).
    >>> 
    >>> 
    >>>> (The NEWS entry is also wrong: The issue existed before readline 6.3)
    >>> 
    >>> Aah.. you are right. The API change with 6.3 was for the other, the
    >>> "SIGWINCH" bug.
    >>> 
    >>> Here's a an update proposal for that NEWS entry :
    >>> 
    >>> ? The API for readline libraries >= 6.3 had changed such
    >>> terminal window resizes where no longer properly signalled
    >>> (PR#16604).  Also, ?Ctrl C? in incremental search behaved
    >>> confusingly in R (unix) consoles (PR#16603) also for older
    >>> readline versions. These have been fixed (for readline >=
    >>> 6.3 only), thanks to patches by Frederick Eaton.
    >>> 
    >>> 
    >>> Martin
    >>> 
    >>>> -pd
    >>>
>>>> On 24 May 2016, at 12:55 , Martin Maechler <maechler at stat.math.ethz.ch> wrote:
>>> 
    >>>>> 
    >>>>> Can you (Frederick, Peter, Keith, but ideally others, too)
    >>>>> confirm that you don't see any problems anymore, when building a
    >>>>> version of R-devel from sources that are newer 
    >>>>> than (or equal to)  svn revision 70632 (2016-05-19 10:59:51, see below)?
    >>>>> 
    >>>>> I'm asking because the question is open if these should be
    >>>>> "back ported" to R 3.3.0 patched or not.
    >>>>> 
    >>>>> Best regards,
    >>>>> Martin
    >>>>> 
    >>>>>>>>>> Martin Maechler <maechler at stat.math.ethz.ch>
    >>>>>>>>>> on Thu, 19 May 2016 11:02:48 +0200 writes:
    >>>>> 
    >>>>>>>>>> <frederik at ofb.net>
    >>>>>>>>>> on Wed, 18 May 2016 15:03:31 -0700 writes:
    >>>>> 
    >>>>>>>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    >>>>>>>> if older versions have trouble finding rl_resize_terminal
    >>>>>>>> then you could wrap a macro around that part.
    >>>>> 
    >>>>>>> I find python related patches that use
    >>>>> 
    >>>>>>> #ifdef HAVE_RL_RESIZE_TERMINAL
    >>>>> 
    >>>>>>> so they must have configured for that.  We could and
    >>>>>>> probably should do the same, but as a Linux_only guy
    >>>>>>> currently (even basically only one flavor of Linux), I'd
    >>>>>>> appreciate others to produce code for that.
    >>>>>
>>>>>
>>>>>
>>>>>
>>>>> 
    >>>>> 
    >>>>> [...........]
    >>> 
    >>>> -- 
    >>>> Peter Dalgaard, Professor,
    >>>> Center for Statistics, Copenhagen Business School
    >>>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
    >>>> Phone: (+45)38153501
    >>>> Office: A 4.23
    >>>> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
    >>>
#
Thank you, Martin. I linked to your message in a comment here so maybe
other people will know about that useful technique:

http://singmann.org/installing-r-devel-on-linux/#comment-161

However, when I try it, I get an error:

    $ make 
    make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/m4'
    make[1]: Nothing to be done for 'R'.
    make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/m4'
    make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/tools'
    make[1]: Nothing to be done for 'R'.
    make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/tools'
    make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc'
    /usr/bin/install: cannot stat '../../R-svn/doc/NEWS': No such file or directory
    /usr/bin/install: cannot stat '../../R-svn/doc/NEWS.pdf': No such file or directory
    Makefile:164: recipe for target 'svnonly' failed
    make[1]: *** [svnonly] Error 1
    make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc'
    Makefile:60: recipe for target 'R' failed
    make: *** [R] Error 1

I configured like this:

    $ cd ../R-svn-build/                                          
    $ ../R-svn/configure --without-recommended-packages --prefix=$HOME/r-svn-test STRIP=true

I guess I can try to debug it myself but thought I should report back
to you. It works when I 'configure' and 'make' in the source
directory.

Cheers,

Frederick
On Tue, May 24, 2016 at 07:20:18PM +0200, Martin Maechler wrote:
#
> Thank you, Martin. I linked to your message in a comment here so maybe
    > other people will know about that useful technique:

    > http://singmann.org/installing-r-devel-on-linux/#comment-161

    > However, when I try it, I get an error:

    > $ make 
    > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/m4'
    > make[1]: Nothing to be done for 'R'.
    > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/m4'
    > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/tools'
    > make[1]: Nothing to be done for 'R'.
    > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/tools'
    > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc'
    > /usr/bin/install: cannot stat '../../R-svn/doc/NEWS': No such file or directory
    > /usr/bin/install: cannot stat '../../R-svn/doc/NEWS.pdf': No such file or directory
    > Makefile:164: recipe for target 'svnonly' failed
    > make[1]: *** [svnonly] Error 1
    > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc'
    > Makefile:60: recipe for target 'R' failed
    > make: *** [R] Error 1

This is strange:  Did you accidentally delete the 'non-tarball'
file in your build directory which should have been created by
'configure' ?

I ask because your 'make' seems to be using the 'else' clause in
the 'svnonly' target in the R-svn-build/Makefile
but it should really use the first branch which does install
things in ./doc/ (such as NEWS or NEWS.pdf).

I have never used  'STRIP=true' -- maybe that did remove the
'non-tarball' file ?

Why not rather do it the way I told you, i.e., *with* recommended
packages, and no arguments to 'configure'
(if that does work, you may try variants.. I agree that
--without-recommended-packages should work as well, I just never
use that).

Martin


    > I configured like this:

    > $ cd ../R-svn-build/                                          
    > $ ../R-svn/configure --without-recommended-packages --prefix=$HOME/r-svn-test STRIP=true

    > I guess I can try to debug it myself but thought I should report back
    > to you. It works when I 'configure' and 'make' in the source
    > directory.

    > Cheers,

    > Frederick
> On Tue, May 24, 2016 at 07:20:18PM +0200, Martin Maechler wrote:
>> >>>>> Keith O'Hara <keith.ohara at nyu.edu>
    >> >>>>>     on Tue, 24 May 2016 12:47:43 -0400 writes:
    >> 
    >> > svn checkout https://svn.r-project.org/R/trunk/ <target-directory>
    >> 
    >> yes, indeed. thank you, Keith.
    >> 
    >> and from then on only
    >> 
    >> cd <target-directory>
    >> svn up
    >> 
    >> (which is short for 'svn update').
    >> 
    >> Another hint: Then do *not* build in the source directory but in
    >> what we called a "build directory"; i.e., something like
    >> (from scratch; including the only-once needed "checkout") :
    >> 
    >> svn checkout https://svn.r-project.org/R/trunk/ R
    >> cd R
    >> tools/rsync-recommended
    >> mkdir ../build-R
    >> cd ../build-R
    >> ../R/configure
    >> make
    >> make check
    >> 
    >> and I then never run 'make install', but rather use symbolic
    >> link from
    >> ..../build-R/bin/R  to something like  ~/bin/R-devel 
    >> i.e.,
    >> cd ~/bin
    >> ln -s ..../build-R/bin/R R-devel
    >> 
    >> Martin
    >>
>> >> On May 24, 2016, at 12:45 PM, frederik at ofb.net wrote:
>> >> 
    >> >> I agree with Martin's summary of the situation, and with the updated
    >> >> NEWS entry.
    >> >> 
    >> >> I'm not familiar with Subversion, can you tell me the command to use?
    >> >> 
    >> >> (I tried "svn co https://svn.r-project.org/R/" but it seems to be
    >> >> downloading all branches)
    >> >> 
    >> >> Frederick
    >> >>
>> >> On Tue, May 24, 2016 at 04:30:11PM +0200, Martin Maechler wrote:
>> >>>>>>>> peter dalgaard <pdalgd at gmail.com>
    >> >>>>>>>> on Tue, 24 May 2016 13:47:27 +0200 writes:
    >> >>> 
    >> >>>> I had a regression in config.site so the nightly build didn't. Retrying....
    >> >>>> Looks like it will build, but the ctl-R, ctl-C bug is still present on OSX (w/Simon's libs). This _was_ fixed for a while, was it not?
    >> >>> 
    >> >>> I thought it was never fixed, for readline versions 5.x (or all
    >> >>> of readline_version < 6.3 ?) because the patch assumed features
    >> >>> not available, e.g., for Frederik (who got compilation errors
    >> >>> which I think you confirmed on pre-6 readline).
    >> >>> 
    >> >>> I remember you having two different readlines installed on OSX
    >> >>> but the standard Mac binary (from CRAN, i.e. Simon) would use
    >> >>> the old readline version ?
    >> >>> 
    >> >>> so that whole  resetReadline() solution is now conditionalized inside
    >> >>> 
    >> >>> #if defined(RL_READLINE_VERSION) && RL_READLINE_VERSION >= 0x0603
    >> >>> ...
    >> >>> ...
    >> >>> #endif
    >> >>> 
    >> >>> and hence the previous code (which is buggy) is used for
    >> >>> readline versions < 6.3.
    >> >>> As a consequence the bug is only fixed for readline >= 6.3,
    >> >>> because the current patch did not compile and hence seemed not
    >> >>> appropriate for  readline < 6.3 (and hence the above conditionalization).
    >> >>> 
    >> >>> 
    >> >>>> (The NEWS entry is also wrong: The issue existed before readline 6.3)
    >> >>> 
    >> >>> Aah.. you are right. The API change with 6.3 was for the other, the
    >> >>> "SIGWINCH" bug.
    >> >>> 
    >> >>> Here's a an update proposal for that NEWS entry :
    >> >>> 
    >> >>> ? The API for readline libraries >= 6.3 had changed such
    >> >>> terminal window resizes where no longer properly signalled
    >> >>> (PR#16604).  Also, ?Ctrl C? in incremental search behaved
    >> >>> confusingly in R (unix) consoles (PR#16603) also for older
    >> >>> readline versions. These have been fixed (for readline >=
    >> >>> 6.3 only), thanks to patches by Frederick Eaton.
    >> >>> 
    >> >>> 
    >> >>> Martin
    >> >>> 
    >> >>>> -pd
    >> >>>
>> >>>> On 24 May 2016, at 12:55 , Martin Maechler <maechler at stat.math.ethz.ch> wrote:
>> >>> 
    >> >>>>> 
    >> >>>>> Can you (Frederick, Peter, Keith, but ideally others, too)
    >> >>>>> confirm that you don't see any problems anymore, when building a
    >> >>>>> version of R-devel from sources that are newer 
    >> >>>>> than (or equal to)  svn revision 70632 (2016-05-19 10:59:51, see below)?
    >> >>>>> 
    >> >>>>> I'm asking because the question is open if these should be
    >> >>>>> "back ported" to R 3.3.0 patched or not.
    >> >>>>> 
    >> >>>>> Best regards,
    >> >>>>> Martin
    >> >>>>> 
    >> >>>>>>>>>> Martin Maechler <maechler at stat.math.ethz.ch>
    >> >>>>>>>>>> on Thu, 19 May 2016 11:02:48 +0200 writes:
    >> >>>>> 
    >> >>>>>>>>>> <frederik at ofb.net>
    >> >>>>>>>>>> on Wed, 18 May 2016 15:03:31 -0700 writes:
    >> >>>>> 
    >> >>>>>>>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
    >> >>>>>>>> if older versions have trouble finding rl_resize_terminal
    >> >>>>>>>> then you could wrap a macro around that part.
    >> >>>>> 
    >> >>>>>>> I find python related patches that use
    >> >>>>> 
    >> >>>>>>> #ifdef HAVE_RL_RESIZE_TERMINAL
    >> >>>>> 
    >> >>>>>>> so they must have configured for that.  We could and
    >> >>>>>>> probably should do the same, but as a Linux_only guy
    >> >>>>>>> currently (even basically only one flavor of Linux), I'd
    >> >>>>>>> appreciate others to produce code for that.
    >> >>>>> 
    >> >>>>> Actually that was easy (in hindsight.. I took too long!)
    >> >>>>> enough, so I've now committed
    >> >>>>> 
    >> >>>>> ------------------------------------------------------------------------
    >> >>>>> r70632 | maechler | 2016-05-19 10:59:51 +0200 (Thu, 19 May 2016) | 1 line
    >> >>>>> Changed paths:
    >> >>>>> M configure
    >> >>>>> M configure.ac
    >> >>>>> M src/include/config.h.in
    >> >>>>> M src/unix/sys-std.c
    >> >>>>> 
    >> >>>>> check for rl_resize_terminal() now
    >> >>>>> ------------------------------------------------------------------------
    >> >>>>> 
    >> >>>>> ... and Keith should even not see the warning anymore
    >> >>>>> (nor Peter the error, when compiling using readline 5.x instead of 6.[23]).
    >> >>>>> 
    >> >>>>> 
    >> >>>>> [...........]
    >> >>> 
    >> >>>> -- 
    >> >>>> Peter Dalgaard, Professor,
    >> >>>> Center for Statistics, Copenhagen Business School
    >> >>>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
    >> >>>> Phone: (+45)38153501
    >> >>>> Office: A 4.23
    >> >>>> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
    >> >>> 
    >>
#
Hi Martin,

Thanks for the Makefile clue. The file 'non-tarball' was present in
the source directory, but not in the build directory (should the
Makefile be checking for 'non-tarball' in the source directory
instead?). However, 'doc/FAQ' was present in the source directory, so
the first clause of '||' failed.

    svnonly:
            @if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \
            ...

It works if I remove 'doc/FAQ'. Incidentally, this file is created by
'make' but not removed by 'make clean'. So the problem is that I
accidentally ran 'make' in the source directory before taking your
advice. I ran 'make clean' out of habit, not even realizing that
'make' in the source directory would break your
separate-build-directory technique.

STRIP=true is highly counterintuitive, but I'm surprised you don't
recognise it - it just replaces the 'strip' command with 'true' so
that my installed binaries get to keep their debugging symbols. Maybe
there is a better way of doing it - I guess you use the binaries in
place so perhaps that is one solution. I'm not sure where I came up
with STRIP=true.




In case anyone skimmed, I think there may be two "action items" here:

1. in Makefile, change

            @if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \

to

            @if test ! -f "$(srcdir)/doc/FAQ" || test -f "$(srcdir)/non-tarball" ; then \

(I haven't tested it)

2. Figure out how to remove doc/FAQ with 'make clean' (if it makes
sense to do so)



Thanks,

Frederick
On Wed, May 25, 2016 at 12:46:53PM +0200, Martin Maechler wrote: