rgdal 1.1-1 bug?

Thank you Tom and Roger for your attention to this issue.

Here is a link to a zipfile (17 MB).  It contains two 
folders/shapefiles.  One is the single big polygon with many holes that 
I had sent to Tom.  The other is the shapefile it was extracted from, 
with many additional polygons showing similar problems.

https://www.dropbox.com/s/xkp8sdoccg6jyg4/offending_shapefile.zip?dl=0

These shapefiles are from Arc and have never been written by writeOGR, 
so should be fine when first opened.

For now, I have been unable to revert to the earlier version of rgdal as 
R complains that it doesn't match the R version.

Jim

-----Original Message-----
From: R-sig-Geo [mailto:r-sig-geo-bounces at r-project.org] On Behalf Of Roger Bivand
Sent: Saturday, December 05, 2015 3:50 AM
To: Tom Philippi
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] rgdal 1.1-1 bug?

On Sat, 5 Dec 2015, Tom Philippi wrote:

James & all--

If you make the offending shapefile available, I'll look at this in a week or so (no time before then). The possible change is from:

https://stat.ethz.ch/pipermail/r-sig-geo/2015-November/023656.html

and affected writing - before Shapefiles were written as-is, from 1.1-1 all output geometries without comments are put through
OGRGeometryFactory::organizePolygons() in GDAL/OGR. This may be a case in which the OGR function fails to organise the polygons in the intended fashion. I have a build of rgdal 1.1-2 for Windows using GDAL
2.0.1 (thanks again to Brian Ripley) which you may try, but it will most likely not change anything. Also, we don't know whether the input data (Polygons objects) have comments or not, but if read through readOGR(), the Shapefile driver also runs OGRGeometryFactory::organizePolygons() on read.

I need access to the offending shapefile to see whether alternative settings to OGRGeometryFactory::organizePolygons(), or a user option to avoid this very sensible function with possibly bad data, might alleviate the problem. Please put on URL or attach offlist.

Roger

With your shapefile I can reproduce your problem under Windows 7x64 in
R
3.2.2 rgdal_1.1-1 sp_1.2-1, but not R 3.1.2 rgdal_0.9-3 sp_1.1-1 or
earlier.  Both rgdals are linked to gdal 1.11.2 released 2015/02/10.
I also can reproduce the issue writing to & reading from GeoJSON
instead of shapefiles.  Files written by rgdal 1.1-1 have the lines
connecting holes when read by rgdal_0.9-3, but files written by
rgdal_0.9-3 are read fine by 1.1-1.

The rarity about the shapefile in question is that it has 57 holes in
the same polygon, which (along with the lines connecting holes)
suggests that the issue might be related to SpatialPolygons not
assigning holes to specific (non-hole) polygons, see:
http://finzi.psych.upenn.edu/R/library/rgeos/html/comment-functions.ht
ml

I don't have time (or ability) to go much further right now in terms
of finding the change, but perhaps someone else can?

Tom 2

On Fri, Dec 4, 2015 at 2:34 PM, Michael Sumner <mdsumner at gmail.com> wrote:

On Sat, 5 Dec 2015 at 05:26 Worrall, James -FS <jworrall at fs.fed.us> wrote:

[image: image001.gif]

We have been happily using rgdal to read and write ESRI Shapefiles.
Recently my colleague began using it.  We noticed that after saving
and reopening files, many polygons had been reconfigured, as if the
order of the vertices had become jumbled and there were many
straight lines cutting across the polygon.  I?m not sure if an image
can be pasted in this list but I?ll try.

I did the same thing on my computer and the problem did not occur.
We found that I had an older version of rgdal, 1.0-7.  When I
updated it to 1.1-1, the problem occurred just as it had with her.

Simply opening and saving a shapefile causes the problem and
essentially jumbles the shapefile.  The bug, if it is one, was
introduced after 1.0-7

I can't reproduce this. Can you provide a reproducible example?
Also, "opening and saving a shapefiles" is not the way to describe
what is done in the usual rgdal workflow, so it's not clear what you
did. Here's what I
tried:

library(maptools)
data(wrld_simpl)

library(rgdal)
#rgdal: version: 1.1-1, (SVN revision 572) # Geospatial Data
Abstraction Library extensions to R successfully loaded # Loaded GDAL
runtime: GDAL 1.11.2, released 2015/02/10 # Path to GDAL shared
files: E:/inst/R/R/library/rgdal/gdal # GDAL does not use iconv for
recoding strings.
# Loaded PROJ.4 runtime: Rel. 4.9.1, 04 March 2015, [PJ_VERSION: 491]
# Path to PROJ.4 shared files: E:/inst/R/R/library/rgdal/proj #
Linking to sp version: 1.2-1


writeOGR(wrld_simpl, ".", "1.1-1", "ESRI Shapefile")
x1 <- readOGR(".", "1.1-1")
plot(x1)  ## all good


 sessionInfo()
R version 3.2.2 Patched (2015-08-27 r69201)
Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 8.1
x64 (build 9600)

locale:
[1] LC_COLLATE=English_Australia.1252
LC_CTYPE=English_Australia.1252 [3]
LC_MONETARY=English_Australia.1252 LC_NUMERIC=C [5]
LC_TIME=English_Australia.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] rgdal_1.1-1     maptools_0.8-37 sp_1.2-1

loaded via a namespace (and not attached):
[1] tools_3.2.2     foreign_0.8-66  grid_3.2.2      lattice_0.20-33


Cheers, Mike.

I will attach images of one complex polygon, before and after saving
with writeOGR.

Jim

	[[alternative HTML version deleted]]

--
Roger Bivand
Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no