Replacement shapefiles with SpatiaLite for R-geo packages(spdep, rgeos etc.)

On Sun, 2 Dec 2012, Edzer Pebesma wrote:

Roger, what do you mean by "Its [Spatialite's] future status is unknown"?

Only that people have been talking it up, but I haven't seen data on adoption 
(or repeated pleas from users) that would show that it has reached a critical 
mass of any kind, let alone of the mass/mess of shapefiles. So for the time 
being, OGR is the obvious route, but building spatialite without Excel, 
without GEOS, etc. I found its compile option to include GEOS development 
(trunk) features very concerning - even GEOS advise against building against 
its experimental features.

A few months ago I found an interesting discussion here:
https://groups.google.com/forum/?hl=en&fromgroups=#!topic/geospatial-mobile-data-format-for-vectors/VEri8doLNz4
which convinced me that spatialite is a software, rather than an open
(file) format - the format is of course there, but not specified
otherwise than by the software that reads/writes it (sqlite).

Well, if it is software, someone could Rcpp it, but I don't see why, when 
what it offers for us is GEOS again, and probably more muddle on CRS and 
codepages - controlling what GDAL/OGR do is a serious enough task without 
mixing in less well-judged and cautious software (QGIS?). My take on the 
discussion you cite would be close to Paul Ramsey's doubts.

I'm also interested in spatialite, as I do not see other really useful
data formats (although it's not a format!) that can be used to exchange
spatio-temporal data to other environments, say ArcGIS or so.

Well, OGR could move in that direction, couldn't it? Then some drivers would 
be have temporal capabilities, others not; of course, someone would need to 
do some work on generalisable data models, but OGR could lead here (INPE?).

Taking the OGR path is of interest, but of course OGR has no idea of
time. For this path, the main burden would be there where people prepare
the distributions that are used by the windows and OS-X CRAN build
trains, right?

That's why contributing upstream to OGR is a better idea, unless the data 
models would become too convoluted. The same applies - as I wrote - to SQL in 
OGR.

The R SQLiteMap package, IIRC, was built on top of the SQLite driver
directly without OGR, and parsed the geometry columns directly; I once
looked at it, but was somehow not overwhelmed by enthousiasm and forgot
why -- was it restricted to WKT representation of geometry?

It was written for a particular purpose some time ago, and is now orphaned by 
its authors. CRAN policy is to orphan maintainerless packages that have 
unanswered issues.

Best wishes,

Roger

Best regards,


On 12/02/2012 03:40 PM, Roger Bivand wrote:

On Sun, 2 Dec 2012, hi_ono2001 at ybb.ne.jp wrote:

Hello.

Recently SpatiaLite 4.0 was released. It has lots of powerful, fast
and robust functions for Geoprocessing(spatial queries, overlay
analysis, creating topology, network analysis, gridding, creating
TIN/voronois). Some people say SpatiaLite is one of candidate to next
standard spatial data format.

Spatialite is supported by OGR in rgdal if the drivers are available.
Its future status is unknown, but of course we should try to make life
easier for its users. If you could document how easy it is to include
the driver in the CRAN Windows and OSX binaries, that would help. In my
opinion, Spatialite should be used through OGR in R, and should not be
used for topological operations, which blunt the sense of having shared
classes in R for spatial data. Using OGR means that things stay
maintainable.

Current rgeos package seems to be slower and unstable for bigger
spatial data. And I think if using SpatiaLite, spdep could create
spatial weight matrices more speedy than current ones. If this, R will
be exactly "R as a GIS".

Since Spatialite uses the same GEOS dependency as rgeos, you should
substaniate this claim with clear reproducible cases; rgeos is now
pretty stable, and misbehaviour now is usually because of erroneous
topologies (overlaps, slivers and dangles). I would strongly advise
against thinking that exporting to Spatialite then GEOS, doing GEOS
there, and importing again, will be faster or more stable than
converting Spatial* objects to GEOS objects, doing GEOS, and converting
back again - the steps are very similar, and such a claim would need to
be studied carefully. The same time could rather be used to contribute
to rgeos. Spatial weights matrices created in spdep can use STRtrees
from GEOS, and are fast (90K polygons in 10 seconds). But this isn't the
default, you need to use extra arguments.

Unfortunately SQLiteMap, R package for accessing SpatiaLite, has been
removed from CRAN since this September.

CRAN has been making its management more systematic - if maintainers do
not respond to requests for corrections, the source packages are
archived until the maintainers comply. This is why maintainability is
crucial. Arguably, the package would also be redundant if OGR Spatialite
drivers were available for those dependent on the CRAN rgdal binaries.

Any plan to employing  this SpatiaLite for current R-geo packages.

Please contribute a vignette to a suitable package, showing how one can
use this storage format with R already. For use of SQL, rather consider
contributing to helping add OGR SQL to readOGR in rgdal, then other
drivers would benefit too.

Best wishes,

Roger

Thanks in advance.