CRS in rgdal does not function with PROJ 6.2.0
On Fri, 8 Nov 2019, Cal Abel wrote:
I have a Ubuntu 18.0.4 with R 3.6.1 system with the following releases from OSGeo for GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0 I have an application that requires using gstat and thus I have to use rgdal as sf objects/classes aren?t supported in gstat, only sp. (This is a bit frustrating because rpostgis uses sf, so when I import my data it is an sf datatype, convert it to sp for gstat, and then back to sf to upload?)
First, note that gstat does support sf and stars class objects, as shown by https://keen-swartz-3146c4.netlify.com/interpolation.html (a draft chapter in a WIP-book). Second, note that unless you are developing software, you should be using GDAL 2 and PROJ 5. You should not be using GDAL 3 and PROJ 6. For references, please do read the links I sent privately explaining why all open source spatial software using GDAL and PROJ is having difficulties adapting to the speed of chane in their APIs. Elsewhere I believe you encountered this with regard to libspatialite. Unless you really need to work on the wild side, stay on the safe side (GDAL2+PROJ5). sf with GDAL3/PROJ6 is also broken now. You are working (day-time job) for a power company, and using R and gstat for free. You need to read more and write less before you impose on volunteer developers and maintainers whose day jobs do not allocate hours to handling your queries.
I thought the issue originally resided in sp, but after reviewing the sp source, it comes down to the calls that the sp CRS function makes using rgdal::. Below is a summary that shows my problem, sf links with the necessary system files and gets the correct CRS, rgdal does not. I think the issue is in how proj_api.h is included, specifically with not recognizing proj.db and instead relying on proj_def.dat which was removed in PROJ 6.1 as I recall. I can confirm that there is only one set of OSGeo libraries installed and those are those are the ones that I?ve compiled and are in /usr/local. I can trick rgdal to use the proj_def.dat file by adding gdal/ and proj/ directories from the latest windows binary *.zip to ~/R/x86_64-pc-linux-gnu-library/3.6/rgdal. After doing this, rgdal will execute CRS, but complains that it can?t find proj.db Once those files are removed it finds proj.db, but then can?t do CRS stuff. Win some, loose some I guess?
This was an issue faced years ago when PROJ was released without this file and is completely misleading here.
After reading through a bit of the source, it seems that rgdal defaults to using the deperecated proj4 strings and does not access the newer formats in proj.db even though during compilation rgdal finds the db with no problem. I am not too familiar with either package, so this is conjecture based on my limited testing/research. As a note, if $PROJ_LIB is specified as an environment variable the result is Path to PROJ.4 shared files: /usr/local/share/proj. But, when it is not specified the result is Path to PROJ.4 shared files: (autodetected). The behavior of rgdal remains unchanged in either case.
library(sf)
Linking to GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0
library(rgdal)
Loading required package: sp rgdal: version: 1.5-1, (SVN revision 879) Geospatial Data Abstraction Library extensions to R successfully loaded Loaded GDAL runtime: GDAL 3.0.1, released 2019/06/28 Path to GDAL shared files: GDAL binary built with GEOS: TRUE Loaded PROJ.4 runtime: Rel. 6.2.0, September 1st, 2019, [PJ_VERSION: 620] Path to PROJ.4 shared files: /usr/local/share/proj Linking to sp version: 1.3-1
CRS("+init=epsg:4269")
Error in CRS("+init=epsg:4269") : no arguments in initialization list
st_crs(4269)
Coordinate Reference System: EPSG: 4269 proj4string: "+proj=longlat +datum=NAD83 +no_defs? Here is a coordinate transformation from NAD83 CRS to the UTM16 NAD83(2011) projection to show basic GDAL/PROJ6 functionality: $ cs2cs EPSG:4269 EPSG:6345 35 -84 773798.10 3877156.69 0.00 I had forgotten that I originally reported the issue here https://github.com/edzer/sp/issues/59 <https://github.com/edzer/sp/issues/59> which I?ve since closed in testing for this email. Below is an email exchange that drove this current email. I am unconvinced that Roger is correct. GDAL/PROJ work just fine, as does sf. A user error that he is referring to would cause those not to work/complain. The fact that I can make it work and make it not work should actually be grounds for successful bug identification.
You did not ask my permission to publish a private exchange on a list posting. It is normal practice to do so. Another time I might rather choose not to reply to any private queries, rather than offer advice, which had you heeded it, would have guided you in the right direction. Check first, read a lot, pester others less, especially when your job depends on our offering goodwill. Reconfigure your system to use GDAL 2 and PROJ 5, the use whatever you need. Roger
Cal
On Wed, 6 Nov 2019, Cal Abel wrote: Dr. Bivand, I attempted to report a bug in rgdal but don't really have any place to put it. I reported it in the OSGeo/gdal git page, but the issue was closed: https://github.com/OSGeo/gdal/issues/1988 <https://github.com/OSGeo/gdal/issues/1988> I posted: Almost certainly user blunder - neither **sf** nor **rgdal** install data/share files on source install, always relying on system installs. OP almost certainly neglected to read install notes. If GDAL and PROJ utilities can find their data/share files outside R, not finding them within R could signal R packages built against a different *.so than the one found at runtime. One update to the above, I tried creating symbolic links to the compiled gdal and proj directories instead of copying the windows folders and it did not work. I also tried various combinations of links/files and was not able to get it to work. The only method that worked was to copy both windows directories. The temporary fix I have, while it does not produce an error it now produces warnings:" Do not try any fixes, you are almost certainly in error. How many versions of GDAL and PROJ are on your platform? For source installs, there should be one only, and GDAL should be built with that PROJ. Warning messages: 1: In CPL_crs_from_epsg(as.integer(x)) : GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db 2: In CPL_crs_from_epsg(as.integer(x)) : GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db " Since it cannot find the PROJ proj.db, your installation of PROJ is broken, and copying across from an older version which does not have this key file will obviously fail. In the future, what is the best avenue for reporting bugs? Use the R-sig-geo list. The package is not on github and will not be migrated - it is where https://cran.r-project.org/package=rgdal <https://cran.r-project.org/package=rgdal> says it is: https://r-forge.r-project.org/projects/rgdal/ <https://r-forge.r-project.org/projects/rgdal/> under SVN. All corresponndence through R-sig-geo, because others can correct your misapprehensions. Note that the nabble link is totally different, which you should have known. Further be aware that your usage pattern (+init=) is no longer valid in GDAL 3 and PROJ 6, and will stop working shortly. Your working example has GDAL 2 and PROJ 5, although this is not the only cause here, I think. It would be to your benefit to read https://proj.org/ <https://proj.org/> and https://gdal.org/ <https://gdal.org/>, and grasp https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html <https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html>, since this is a work-related query. How you migrate your corporate workflows to GDAL 3 and PROJ 6//7 may be of considerable interest (blog, mailing list post), and I'm spending all my time trying to adapt rgdal for this major change, and do not have patience with wasted time. See https://github.com/r-spatial/sf/issues/1146 <https://github.com/r-spatial/sf/issues/1146> and https://github.com/r-spatial/discuss/issues/28 <https://github.com/r-spatial/discuss/issues/28>. Roger Bivand
Cal Abel, PhD crabel at signalpowerandlight.com <mailto:crabel at signalpowerandlight.com> www.signalpowerandlight.com <http://www.signalpowerandlight.com/>
Roger Bivand Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no https://orcid.org/0000-0003-2392-6140 https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en