Skip to content

Errors at installing rgdal on Debian Buster

8 messages · Edzer Pebesma, Agustin Lobo, Roger Bivand

#
The problem seems to be related to the compiler. I had:
gcc version 5.3.1 20160101 (Debian 5.3.1-5)

I have now:
gcc version 7.2.0 (Debian 7.2.0-4)

All weird previus errors are gone but I still have:
configure: PROJ.4 version: 4.8.0
./configure: line 3725:  8390 Segmentation fault      ./proj_conf_test
checking PROJ.4: epsg found and readable... yes
./configure: line 3800:  8399 Segmentation fault      ./proj_conf_test

and finally

** testing if installed package can be loaded
Fatal error: glibc detected an invalid stdio handle
Aborted
ERROR: loading failed
* removing ?/home/alobo/R/i686-pc-linux-gnu-library/3.3/rgdal?

Surprisingly,  I do have 4.9 installed, not 4.8:

$ dpkg -s proj-bin
Package: proj-bin
Status: install ok installed
Priority: optional
Section: science
Installed-Size: 125
Maintainer: Debian GIS Project <pkg-grass-devel at lists.alioth.debian.org>
Architecture: i386
Source: proj
Version: 4.9.3-2

and
$ proj
Rel. 4.9.3, 15 August 2016
usage: proj [ -bCeEfiIlormsStTvVwW [args] ] [ +opts[=arg] ] [ files ]

and I cannot find proj_conf_test anywhere:
$ sudo find / -name 'proj_conf_test'
gives no result

How is the test ./proj_conf_test done so that I can investigate why it
results into PROJ 4.8 instead of 4.9?

Thanks

Agus
On Tue, Sep 12, 2017 at 1:49 PM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
#
IMHO, debugging a problem like this would become so much easier if you
could provide a docker file that creates the machine you are looking at.
Other docker users (like me) could then run it, reproduce your machine,
and look at what is going wrong.

A set of r-spatial targeting rocker (debian)-based docker images are
found here:

https://github.com/rocker-org/geospatial

a set of ubuntu-based docker files are found here:

https://github.com/r-spatial/sf/tree/master/inst/docker
On 13/09/17 09:37, Agustin Lobo wrote:

  
    
#
On Wed, 13 Sep 2017, Agustin Lobo wrote:

            
It is defined in rgdal/configure.ac (line 273 ff.):

if test ${proj_version} -ge 480; then
[cat > proj_conf_test.c <<_EOCONF
#include <stdio.h>
#include <proj_api.h>
#if PJ_VERSION == 480
FILE *pj_open_lib(projCtx, const char *, const char *);
#endif

int main() {
#if PJ_VERSION <= 480
     FILE *fp;
#else
     PAFile fp;
#endif
     projCtx ctx;
     ctx = pj_get_default_ctx();
     fp = pj_open_lib(ctx, "epsg", "rb");
     if (fp == NULL) exit(1);
#if PJ_VERSION <= 480
     fclose(fp);
#else
     pj_ctx_fclose(ctx, fp);
#endif
     exit(0);
}
_EOCONF]
else
[cat > proj_conf_test.c <<_EOCONF
#include <stdio.h>
#include <proj_api.h>
FILE *pj_open_lib(const char *, const char *);

int main() {
     FILE *fp;
     fp = pj_open_lib("epsg", "rb");
     if (fp == NULL) exit(1);
     fclose(fp);
     exit(0);
}
_EOCONF]
fi

and checks that the crucial epsg file is present. It does seem to run, 
though, which is puzzling - if you read configure.ac, you'll see that I 
didn't guard against the previous (different) ./proj_conf_test already 
existing - I'll revise the test names to avoid such false positives in the 
future.

Roger

  
    
#
As Edzer said earlier in this thread (and Agus was quite right to post 
rather than contact me directly), reproducing the problem(s) is hard 
without access to the specific platform (thread started here):

https://stat.ethz.ch/pipermail/r-sig-geo/2017-September/025974.html

My guesses so far are that the gcc build trains used for R, and for the 
GDAL/PROJ.4 components are not the same, and may not be the same as the 
installed gcc build train used to install rgdal from source. This probably 
also affects sf on debian buster (and ubuntu?). It may also affect 
GEOS/rgeos.

The gcc versions do change as platforms are upgraded, so for anything 
using C++, the ABI changes will bite hard, and keeping the compiler 
versions in harmony is crucial. The baseline comparison is to install GDAL 
and its dependencies (including PROJ.4) from source (download and unpack 
source tarball, ./configure && make && make install), then probably R from 
source, finally rgdal. This has to work, but unpicks the debian/ubuntu 
packaging system on which many rely.

Please also consider taking this up on r-sig-debian, with a reproducible 
example, best in a docker container.

A further thought is that the most frequent cause of trouble on 
debian/ubuntu previously were multiple installs of different versions of 
GDAL, pulled in by different downstream packages, but the current reports 
do not suggest this as the cause. Still worth checking, though ...

Roger
On Wed, 13 Sep 2017, Roger Bivand wrote:

            

  
    
#
I'm now very confused. It seems to me, in line with what you say, that
I had 2 different
kinds of problems:
- Those derived from upgrading from Debian Stretch to Buster (a
consequence of keeping the system as Debian testing: it is just the
consequence of testing passing from Stretch to Buster as
Stretch becomes Stable) and keeping some older versions of different
components (probably installed by packages outside Debian repos). I
guess these problems have been fixed (although I still do not
understand why the rgdal installation claims I have PROJ4.8, which I
cannot find in my system).
- Those derived from Debian packages having been compiled with
different and incompatible options. This is something I cannot fix
myself.

Therefore:

- I will go on compiling myself as Roger suggests to regain an
operational system.
- I will try to test on a fresh Buster system (or a docker, which I
will have to find out what it is...)
- I will report to r-sig-debian and gdal-dev (please suggest an
alternative gdal linst if
gdal-dev is not appropriate).

I'll keep this list informed.

Thanks!

Agus
On Thu, Sep 14, 2017 at 9:26 AM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
#
On Fri, 15 Sep 2017, Agustin Lobo wrote:

            
Please see whether there are multiple proj_api.h files on your system - if 
so, rgdal is picking up the first in your search path, but may link 
against a later *.so, explaining the crashes during the configure run.
https://github.com/rocker-org/rocker
http://dirk.eddelbuettel.com/papers/useR2015_docker.pdf

for example.
(please keep your heads covered as GEOS and GDAL transition to requiring
others can learn from hard-won experience.

Roger

  
    
2 days later
#
rgdal compiles fine on my Debian Testing (Buster) system by removing
an older pro_api.h file:

1. As suggested by Roger, I located and removed an older version of
proj_api.h taht had not been removed when I removed the older proj4
version:
$ sudo find / -name proj_api.h
/usr/local/include/proj_api.h
/usr/include/proj_api.h
alobo at debi:~$ sudo rm /usr/local/include/proj_api.h

2. Then the rgdal installation goes smoothly;

$ R CMD INSTALL rgdal_1.2-8.tar.gz
.../...
configure: PROJ.4 version: > 4.8.0
.../...

(see https://www.dropbox.com/s/9i8rk74k0vk95uc/rgdal2.log?dl=0 for the
entire output)

3. Then, in R, rgdal loads with no problems
Loading required package: rgdal
Loading required package: sp
rgdal: version: 1.2-8, (SVN revision 663)
 Geospatial Data Abstraction Library extensions to R successfully loaded
 Loaded GDAL runtime: GDAL 2.2.1, released 2017/06/23
 Path to GDAL shared files: /usr/share/gdal/2.2
 Loaded PROJ.4 runtime: Rel. 4.9.3, 15 August 2016, [PJ_VERSION: 493]
 Path to PROJ.4 shared files: (autodetected)
 Linking to sp version: 1.2-5
R version 3.3.3 (2017-03-06)
Platform: i686-pc-linux-gnu (32-bit)
Running under: Debian GNU/Linux buster/sid

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] rgdal_1.2-8 sp_1.2-5

loaded via a namespace (and not attached):
[1] tools_3.3.3     grid_3.3.3      lattice_0.20-35

4. Thus I guess all problems were caused by an old gcc compiler +
leftovers in /usr/local/... directories and that I should
report to gdal-dev and r-sig-debian that there is actually no problem
with versions of R. gfal and proj4 in the Debian testing (Buster)
repos.

Correct?

Agus
On Fri, Sep 15, 2017 at 9:32 AM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
#
On Mon, 18 Sep 2017, Agustin Lobo wrote:

            
Thanks for reporting back! Unless you posted questions on those list, then 
your posting here should be sufficient, as the list is searchable.

Roger