gfortran 9 quantreg bug

With extensive help from Dirk Eddelbuettel I have installed 
docker on my mac mini from 

    https://hub.docker.com/editions/community/docker-ce-desktop-mac

which installs from a dmg in quite standard fashion.  This has allowed
me to simulate running R in a Debian environment with gfortran-9 and
begin the process of debugging my ancient rqbr.f code.

Some further details:

0.  After some initial testing, e.g.

    docker --version 
    docker run hello-world       

1.  Download r-base and test os

    docker pull r-base                       $ downloads r-base for us 
    docker run --rm -ti r-base R --version   # to check we have the R we want
    docker run --rm -ti r-base bash          # now in shell, Ctrl-d to exit

2.  Setup working directory -- tell Docker to run from the current directory
    and access

    cd projects/rq
    docker run --rm -ti -v ${PWD}:/work -w /work r-base bash

    This put the contents of projects/rq into the /work directory.

    root at 90521904fa86:/work# apt-get update
    Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB]
    Get:2 http://cdn-fastly.deb.debian.org/debian testing InRelease [117 kB]
    Get:3 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,385 kB]
    Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 Packages [7,916 kB]
    Fetched 16.6 MB in 4s (4,411 kB/s)                           
    Reading package lists... Done
  
3.  Get gcc-9 and gfortran-9

    root at 90521904fa86:/work# apt-get install gcc-9 gfortran-9
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following additional packages will be installed:
    cpp-9 gcc-9-base libasan5 libatomic1 libcc1-0 libgcc-9-dev libgcc1 libgfortran-9-dev
    libgfortran5 libgomp1 libitm1 liblsan0 libquadmath0 libstdc++6 libtsan0 libubsan1
    Suggested packages:
    gcc-9-locales gcc-9-multilib gcc-9-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg
    libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libquadmath0-dbg gfortran-9-multilib
    gfortran-9-doc libgfortran5-dbg libcoarrays-dev
    The following NEW packages will be installed:
    cpp-9 gcc-9 gfortran-9 libgcc-9-dev libgfortran-9-dev
    The following packages will be upgraded:
    gcc-9-base libasan5 libatomic1 libcc1-0 libgcc1 libgfortran5 libgomp1 libitm1 liblsan0
    libquadmath0 libstdc++6 libtsan0 libubsan1
    13 upgraded, 5 newly installed, 0 to remove and 71 not upgraded.
    Need to get 35.6 MB of archives.
    After this operation, 107 MB of additional disk space will be used.
    Do you want to continue? [Y/n] Y
    Get:1 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libasan5 amd64 9.1.0-10 [390 kB]
    Get:2 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libubsan1 amd64 9.1.0-10 [128 kB]
    Get:3 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libtsan0 amd64 9.1.0-10 [295 kB]
    Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9-base amd64 9.1.0-10 [190 kB]
    Get:5 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libstdc++6 amd64 9.1.0-10 [500 kB]
    Get:6 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libquadmath0 amd64 9.1.0-10 [145 kB]
    Get:7 http://cdn-fastly.deb.debian.org/debian testing/main amd64 liblsan0 amd64 9.1.0-10 [137 kB]
    Get:8 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libitm1 amd64 9.1.0-10 [27.6 kB]
    Get:9 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgomp1 amd64 9.1.0-10 [88.1 kB]
    Get:10 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran5 amd64 9.1.0-10 [633 kB]
    Get:11 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libcc1-0 amd64 9.1.0-10 [47.7 kB]
    Get:12 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libatomic1 amd64 9.1.0-10 [9,012 B]
    Get:13 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc1 amd64 1:9.1.0-10 [40.5 kB]
    Get:14 http://cdn-fastly.deb.debian.org/debian testing/main amd64 cpp-9 amd64 9.1.0-10 [9,667 kB]
    Get:15 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc-9-dev amd64 9.1.0-10 [2,346 kB]
    Get:16 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9 amd64 9.1.0-10 [9,945 kB]
    Get:17 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran-9-dev amd64 9.1.0-10 [676 kB]
    Get:18 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gfortran-9 amd64 9.1.0-10 [10.4 MB]
    Fetched 35.6 MB in 6s (6,216 kB/s)      
    debconf: delaying package configuration, since apt-utils is not installed
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../libasan5_9.1.0-10_amd64.deb ...
    Unpacking libasan5:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../libubsan1_9.1.0-10_amd64.deb ...
    Unpacking libubsan1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../libtsan0_9.1.0-10_amd64.deb ...
    Unpacking libtsan0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../gcc-9-base_9.1.0-10_amd64.deb ...
    Unpacking gcc-9-base:amd64 (9.1.0-10) over (9.1.0-8) ...
    Setting up gcc-9-base:amd64 (9.1.0-10) ...
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../libstdc++6_9.1.0-10_amd64.deb ...
    Unpacking libstdc++6:amd64 (9.1.0-10) over (9.1.0-8) ...
    Setting up libstdc++6:amd64 (9.1.0-10) ...
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../0-libquadmath0_9.1.0-10_amd64.deb ...
    Unpacking libquadmath0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../1-liblsan0_9.1.0-10_amd64.deb ...
    Unpacking liblsan0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../2-libitm1_9.1.0-10_amd64.deb ...
    Unpacking libitm1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../3-libgomp1_9.1.0-10_amd64.deb ...
    Unpacking libgomp1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../4-libgfortran5_9.1.0-10_amd64.deb ...
    Unpacking libgfortran5:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../5-libcc1-0_9.1.0-10_amd64.deb ...
    Unpacking libcc1-0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../6-libatomic1_9.1.0-10_amd64.deb ...
    Unpacking libatomic1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../7-libgcc1_1%3a9.1.0-10_amd64.deb ...
    Unpacking libgcc1:amd64 (1:9.1.0-10) over (1:9.1.0-8) ...
    Setting up libgcc1:amd64 (1:9.1.0-10) ...
    Selecting previously unselected package cpp-9.
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../cpp-9_9.1.0-10_amd64.deb ...
    Unpacking cpp-9 (9.1.0-10) ...
    Selecting previously unselected package libgcc-9-dev:amd64.
    Preparing to unpack .../libgcc-9-dev_9.1.0-10_amd64.deb ...
    Unpacking libgcc-9-dev:amd64 (9.1.0-10) ...
    Selecting previously unselected package gcc-9.
    Preparing to unpack .../gcc-9_9.1.0-10_amd64.deb ...
    Unpacking gcc-9 (9.1.0-10) ...
    Selecting previously unselected package libgfortran-9-dev:amd64.
    Preparing to unpack .../libgfortran-9-dev_9.1.0-10_amd64.deb ...
    Unpacking libgfortran-9-dev:amd64 (9.1.0-10) ...
    Selecting previously unselected package gfortran-9.
    Preparing to unpack .../gfortran-9_9.1.0-10_amd64.deb ...
    Unpacking gfortran-9 (9.1.0-10) ...
    Setting up libgomp1:amd64 (9.1.0-10) ...
    Setting up libasan5:amd64 (9.1.0-10) ...
    Setting up libquadmath0:amd64 (9.1.0-10) ...
    Setting up libatomic1:amd64 (9.1.0-10) ...
    Setting up libgfortran5:amd64 (9.1.0-10) ...
    Setting up libubsan1:amd64 (9.1.0-10) ...
    Setting up cpp-9 (9.1.0-10) ...
    Setting up libcc1-0:amd64 (9.1.0-10) ...
    Setting up liblsan0:amd64 (9.1.0-10) ...
    Setting up libitm1:amd64 (9.1.0-10) ...
    Setting up libtsan0:amd64 (9.1.0-10) ...
    Setting up libgcc-9-dev:amd64 (9.1.0-10) ...
    Setting up gcc-9 (9.1.0-10) ...
    Setting up libgfortran-9-dev:amd64 (9.1.0-10) ...
    Setting up gfortran-9 (9.1.0-10) ...
    Processing triggers for libc-bin (2.28-10) ...
    root at 90521904fa86:/work# pwd
    
4.  At this point I removed some dependencies from the package
    quantreg that I knew were not relevant to the debugging problem at
    hand. This included an attempt to set compiler flags in
    quantreg/src/Makevars, but this didn't work.

5.  Set compiler flags as follows:

    root at 90521904fa86:/work# mkdir ~/.R; vi ~/.R/Makevars 

	CC=gcc-9
	FC=gfortran-9
	F77=gfortran-9

    Alternatively, one can find the settings of CC, FC, CXX, ... in /etc/R/Makeconf
    and alter them there.
    
6.  At this point 

    R CMD INSTALL quantreg_5.43.tar.gz

    did use the gfortran-9  compiler and this version did reproduce the 
    error initially reported by Kurt and Brian.  

7.  Now it is (just!) a matter of finding the bug.
     
Roger Koenker
r.koenker at ucl.ac.uk
Department of Economics, UCL
London  WC1H 0AX.

On Aug 4, 2019, at 3:41 PM, Dirk Eddelbuettel <edd at debian.org> wrote:


Roger,

On 4 August 2019 at 06:48, Koenker, Roger W wrote:
| I?d like to solicit some advice on a debugging problem I have in the quantreg package.
| Kurt and Brian have reported to me that on Debian machines with gfortran 9
| 
| library(quantreg)
| f = summary(rq(foodexp ~ income, data = engel, tau = 1:4/5))
| plot(f)
| 
| fails because summary() produces bogus estimates of the coefficient bounds.
| This example has been around in my R package from the earliest days of R, and
| before that in various incarnations of S.  The culprit is apparently rqbr.f which is
| even more ancient, but must have something that gfortran 9 doesn?t approve of.
| 
| I note that in R-devel there have been some other issues with gfortran 9, but these seem
| unrelated to my problem.  Not having access to a machine with an R/gfortran9
| configuration, I can?t  apply my rudimentary debugging methods.  I?ve considered
| trying to build gfortran on my mac air and then building R from source, but before
| going down this road, I wondered whether others had other suggestions, or
| advice about  my proposed route.  As far as I can see there are not yet
| binaries for gfortran 9 for osx.

Maybe installing and running Docker on your mac is an alternative?

Minimally viable example using

 a) docker (on Linux, but it is portable) and

 b) the current official 'r-base' container (an alias to our Rocker r-base container)

r-base is begged to Debian testing, and also allows you to get Debian
unstable.  Below I fire up the container, tell it to use bash (not R) and update

 edd at rob:~/git$ docker run --rm -ti r-base bash
 root at 1307193fadf4:/# 
 root at 1307193fadf4:/# apt-get update
 Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB]
 Get:2 http://cdn-fastly.deb.debian.org/debian testing InRelease [117 kB]
 Get:3 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,385 kB]
 Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 Packages [7,918 kB]
 Fetched 16.6 MB in 4s (4,649 kB/s)                           
 Reading package lists... Done
 root at 1307193fadf4:/# apt-cache policy gcc-9
 gcc-9:
   Installed: (none)
   Candidate: 9.1.0-10
   Version table:
      9.1.0-10 990
         990 http://deb.debian.org/debian testing/main amd64 Packages
         500 http://http.debian.net/debian sid/main amd64 Packages
 root at 1307193fadf4:/# apt-cache policy gfortran-9
 gfortran-9:
   Installed: (none)
   Candidate: 9.1.0-10
   Version table:
      9.1.0-10 990
         990 http://deb.debian.org/debian testing/main amd64 Packages
         500 http://http.debian.net/debian sid/main amd64 Packages
 root at 1307193fadf4:/# 

At this point it just a matter of actually installing gcc-9 and gfortran-9
(via apt-get install ...), and setting CC, FC, F77 and whichever other
environment variables the R build reflect to build quantreg.

That said, this will be Debian's standard gfortran-9.  What is at times a
little frustrating is that some of the builds used by some of the CRAN tests
use local modifications which make their behaviour a little harder to
reproduce.  I have an open issue with my (also old and stable) digest package
which goes belly-up on a clang-on-Fedora build and nowhere else -- and I have
been unable to reproduce this too.

For such cases, having Docker container would be one possible way of
giving access to the test environment to make it accessible to more users.

Best,  Dirk


| 
| Thanks,
| Roger
| 
| Roger Koenker
| r.koenker at ucl.ac.uk<mailto:r.koenker at ucl.ac.uk>
| Department of Economics, UCL
| London  WC1H 0AX.
| 
| 
| 
| 	[[alternative HTML version deleted]]
| 
| ______________________________________________
| R-devel at r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org