Skip to content

Replacement shapefiles with SpatiaLite for R-geo packages(spdep, rgeos etc.)

5 messages · Hisaji ONO, Edzer Pebesma, Roger Bivand

Original
Hisaji ONO
# 
Hello.

 Recently SpatiaLite 4.0 was released. It has lots of powerful, fast and robust functions for Geoprocessing(spatial queries, overlay analysis, creating topology, network analysis, gridding, creating TIN/voronois).
 Some people say SpatiaLite is one of candidate to next standard spatial data format. 

 Current rgeos package seems to be slower and unstable for bigger spatial data.
 And I think if using SpatiaLite, spdep could create spatial weight matrices more speedy than current ones. If this, R will be exactly "R as a GIS".

Unfortunately SQLiteMap, R package for accessing SpatiaLite, has been removed from CRAN since this September.

Any plan to employing  this SpatiaLite for current R-geo packages.

Thanks in advance.
Roger Bivand
#
On Sun, 2 Dec 2012, hi_ono2001 at ybb.ne.jp wrote:

            

      
      
      
    

        
Spatialite is supported by OGR in rgdal if the drivers are available. Its 
future status is unknown, but of course we should try to make life easier 
for its users. If you could document how easy it is to include the driver 
in the CRAN Windows and OSX binaries, that would help. In my opinion, 
Spatialite should be used through OGR in R, and should not be used for 
topological operations, which blunt the sense of having shared classes in 
R for spatial data. Using OGR means that things stay maintainable.

            

      
      
      
    

        
Since Spatialite uses the same GEOS dependency as rgeos, you should 
substaniate this claim with clear reproducible cases; rgeos is now pretty 
stable, and misbehaviour now is usually because of erroneous topologies 
(overlaps, slivers and dangles). I would strongly advise against thinking 
that exporting to Spatialite then GEOS, doing GEOS there, and importing 
again, will be faster or more stable than converting Spatial* objects to 
GEOS objects, doing GEOS, and converting back again - the steps are very 
similar, and such a claim would need to be studied carefully. The same 
time could rather be used to contribute to rgeos. Spatial weights matrices 
created in spdep can use STRtrees from GEOS, and are fast (90K polygons in 
10 seconds). But this isn't the default, you need to use extra arguments.

            

      
      
      
    

        
CRAN has been making its management more systematic - if maintainers do 
not respond to requests for corrections, the source packages are archived 
until the maintainers comply. This is why maintainability is crucial. 
Arguably, the package would also be redundant if OGR Spatialite drivers 
were available for those dependent on the CRAN rgdal binaries.

            

      
      
      
    

        
Please contribute a vignette to a suitable package, showing how one can 
use this storage format with R already. For use of SQL, rather consider 
contributing to helping add OGR SQL to readOGR in rgdal, then other 
drivers would benefit too.

Best wishes,

Roger

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Edzer Pebesma
    

    

      
      #
    

  


  

      
Roger, what do you mean by "Its [Spatialite's] future status is unknown"?

A few months ago I found an interesting discussion here:
https://groups.google.com/forum/?hl=en&fromgroups=#!topic/geospatial-mobile-data-format-for-vectors/VEri8doLNz4
which convinced me that spatialite is a software, rather than an open
(file) format - the format is of course there, but not specified
otherwise than by the software that reads/writes it (sqlite).

I'm also interested in spatialite, as I do not see other really useful
data formats (although it's not a format!) that can be used to exchange
spatio-temporal data to other environments, say ArcGIS or so.

Taking the OGR path is of interest, but of course OGR has no idea of
time. For this path, the main burden would be there where people prepare
the distributions that are used by the windows and OS-X CRAN build
trains, right?

The R SQLiteMap package, IIRC, was built on top of the SQLite driver
directly without OGR, and parsed the geometry columns directly; I once
looked at it, but was somehow not overwhelmed by enthousiasm and forgot
why -- was it restricted to WKT representation of geometry?

Best regards,

        
On 12/02/2012 03:40 PM, Roger Bivand wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Sun, 2 Dec 2012, Edzer Pebesma wrote:

        

            

      
      
      
    

        
Only that people have been talking it up, but I haven't seen data on 
adoption (or repeated pleas from users) that would show that it has 
reached a critical mass of any kind, let alone of the mass/mess of 
shapefiles. So for the time being, OGR is the obvious route, but building 
spatialite without Excel, without GEOS, etc. I found its compile option to 
include GEOS development (trunk) features very concerning - even GEOS 
advise against building against its experimental features.

            

      
      
      
    

        
Well, if it is software, someone could Rcpp it, but I don't see why, when 
what it offers for us is GEOS again, and probably more muddle on CRS and 
codepages - controlling what GDAL/OGR do is a serious enough task without 
mixing in less well-judged and cautious software (QGIS?). My take on the 
discussion you cite would be close to Paul Ramsey's doubts.

            

      
      
      
    

        
Well, OGR could move in that direction, couldn't it? Then some drivers 
would be have temporal capabilities, others not; of course, someone would 
need to do some work on generalisable data models, but OGR could lead 
here (INPE?).

            

      
      
      
    

        
That's why contributing upstream to OGR is a better idea, unless the data 
models would become too convoluted. The same applies - as I wrote - to SQL 
in OGR.

            

      
      
      
    

        
It was written for a particular purpose some time ago, and is now orphaned 
by its authors. CRAN policy is to orphan maintainerless packages that have 
unanswered issues.

Best wishes,

Roger

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Mon, 3 Dec 2012, Roger Bivand wrote:

        

            

      
      
      
    

        
In addition, in trying to see the benefits of the file format, I loaded:

https://groups.google.com/group/spatialite-users/attach/9fd2f47b1a686829/meridian_test.7z?part=4&view=1

(3.6 MB) into R as three layers (rivers, rail, woods). In R, the sum of 
the layers is 20KB. Am I missing something, or is the format bloated? 
Maybe the choice made by GADM to distribute in RData format makes more 
sense? GDAL already supports SpatialGridDataFrame objects.

Roger