Pre-GDAL 2: rgdal changes - from all according to their means

On Tue, 2 Jun 2015, chris english wrote:

Roger,

I second Michael Sumner's excitement. I built a POSTGist SFCGAL stack

based

on GDAL 2.0.0beta, anticipating that I could seamlessly integrate with R
and was disappointed that I couldn't use  2.0.0.beta  because at the

point

only sp-1.1-1 was supported.

Chris,

Excitement doesn't help. GDAL 2 is only an advance over GDAL 1 in that it
makes the code base easier to maintain because only the GDAL driver
manager is supported, and vector drivers have been folded into the same
model. Beyond that, we have continuing feature creep (e.g. use of
Integer64 to hold IDs that notionally should be strings (no arithmetic)
and which always lose leading zeros, and others. The key challenge
(threat) to GDAL is the very limited number of core developers, followed
by the same danger affecting many of GDAL's external dependencies.

General comment: Could someone who knows how to do it please set up a
searchable scribble sheet to which ordered examples, open issues, ideas,
etc. could be contributed? This thread is now the prime location, but
probably will become unwieldy.

You will never be able to switch easily between rgdal installed against
GDAL 1 and rgdal installed against GDAL 2. Any possibilities are only
available to those installing GDAL from source, then rgdal from source, as
GDAL 2 overwrites GDAL 1 (same name for gdal-config, same app names) if
--prefix= is the same. If --prefix= is different between 1 and 2 when
installing GDAL, and you take care with LD path names and use of ldconfig,
you can pass these different locations of GDAL 1 and GDAL 2 through when
installing rgdal. Then you'd also have to remember to install rgdal for
GDAL 1 in say one R library folder, for GDAL 2 in a different library
folder, and add the appropriate one to .libPaths(). It would be messy.

Performance-wise, there should be no user-impacting difference between
GDAL 1 and 2, or between rgdal opening for GDAL 1 or 2, and rgdal (<=
0.9-3), which blocks GDAL 2.

The task now is to locate possible differences in behaviour between rgdal
0.9-3 with GDAL 1, and rgdal 1.0-2 with either GDAL 1 or 2. If there are
1|2 differences, are they because of changes in GDAL, or because rgdal was
making inappropriate assumptions about GDAL?

So far, we really don't know the Integer64 field type in GDAL 2 vectors is
going to play out - as of now rgdal 1.0-2 truncates to integer. We don't
know how the changes in the driver manager for vector dsn and layers
affect layer and dsn overwriting (I'm seeing very odd OGRErr numbers in
GDAL 2, and apparently different behaviour for some drivers). Writing
MapInfo File TAB files looks broken (maybe a dsn creation option needs
setting, when it wasn't needed before), etc.

Some of this can be fixed by nasty kludges in rgdal, some should be fixed
in GDAL 2, some are real misunderstandings. I don't have the time to try
to make an rgdal Windows binary built against Windows GDAL 2 by
cross-compiling for trying things out under Windows without user
compilation. The transition will take time, but anyone who can install
GDAL and rgdal from source, and who is interested and/or needs rgdal to
work predictably, is welcome to run some standard procedures with rgdal
0.9-3 and GDAL 1, storing a text version of key results; install rgdal
1.0-2 (or later) and GDAL 1 (or later checked out from R-Forge and living
under pkg/:

#cd rgdal
R CMD build pkg
R CMD check rgdal_1.0-2.tar.gz
# maybe --install-args='--configure.args=...'
R CMD INSTALL rgdal_1.0-2.tar.gz
# maybe --configure.args=...

rerun the procedure and compare the results to see whether changes in
rgdal have led to changes in the results, the re-install rgdal 1.0-2 with
GDAL 2, rerun the procedure and compare the results, and compare those for
GDAL 1 and GDAL 2.

In particular, we need confirmation that the database access drivers work
across these shifts.

Roger

I have a couple of future projects that are going to fundamentally depend
on GDAL and I hope to be able to address them from within R.  As an R
beginner I'm happy I have in my notes on how to point to my /opt/gdal(
please excuse if generalizing between an installed /opt/gdal(2.0,0.beta

and

/usr/local/gdal(earlier.build)  build, but many people perhaps have not
been making a diary of R success(es).

In the simplest case it could be a reminder to us beginners how we might
load a library(rgdal("in this case 2.0.0.beta vs our normal
library(rgdal)).  Well we are beginners, but we still want out stuff to
work and we want to contribute to a successful and comprehensive 2.2.0
release. So we'll run our particular research examples through 2.0.0 and
report results, if we know how to load one vs. another library.

If intermediate or expert, different examples would be deployed.

I apologize in advance if these issues  or methods have already been
addressed at SVN and I failed to notice them.  If that is the case I will
dig further, and please ignore me.

I think GDAL and rgdal are intrinsically important tools to this

community

and everyone wants them to work; please just tell us, by general example,
how we might be best of service.

I hope the foregoing is comprehensible. My thoughts, and looking forward

to

to contributing in a useful fashion to this important package (as a
beginner).

Cheers,

Chris


The second beta of GDAL 2 is now available, and as of revision 535 on

R-Forge, the legacy rgdal package passes R CMD check with either GDAL
1.11.2 (or earlier) or GDAL 2.0.0 beta 2.

One or two issues are known (Integer64 in vector fields and FIDs not
supported in R; gdalDrivers() reports both raster and vector drivers;

the

MapInfo File TAB driver doesn't work for writing, ...), but others

remain

to be discovered.

For those who need rgdal in production, and can install the 2.0.0 beta,

it

would be a really good use of time to identify issues now, rather than

when

GDAL 2 starts to become the standard, stable release. Anyone else

needing

an itch to scratch is also, of course, welcome to contribute.

The rgdal package will continue to condition on GDAL 1 or 2, so

hopefully

those users who do not need to move to GDAL 2 will not be affected.
However, it is worth noting that GDAL is maintained by very, very, few
volunteers (even plural is questionable here), and when they feel that
backporting fixed from GDAL 2 to GDAL 1 is taking time from more

important

things, you will be stranded with EOL software.

So please consider taking the time to contribute to the idenfication of
issues in the development version of rgdal built against GDAL 2 and/or

1,

available for anonymous SVN checkout at:

svn checkout svn://scm.r-forge.r-project.org/svnroot/rgdal/trunk

Enjoy!

Roger

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no