Skip to content

rgdal_1.1-1: dsn path string problems

6 messages · Rainer Hurling, Roger Bivand

Original
Rainer Hurling
# 
Dear Roger, dear list,

many thanks for the newest update 1.1-1 of package rgdal. It is really
helpful.

I am using rgdal with R-devel and GDAL 2.0.1, both on FreeBSD
11.0-CURRENT and Windows7. On both platforms, I have problems with path
strings in dsn.



(a) The minor problem is on Windows. While a dsn="C:/some/path" works
fine, the same path with a slash at the end (dsn="C:/some/path/") fails.

On most other software it is conventional to allow a closing slash.
Wouldn't it be nice, if rgdal would be more tolerant about it?


Now my main problem:
(b) On FreeBSD, for some time now, the dsn path can not be used anymore
to check for existence of a file.

I want to use writeOGR to overwrite an existing shapefile. I use
something like dsn="/path/to/somewhere" and layer="layername".

Within writeOGR(), R/ogr_write.R calls the C++ function
'ogrCheckExists'. ogrCheckExists is not able to return TRUE, if a
shapefile already exists. Every time its result is FALSE:

R/ogr_write.R
61:        ogrI <- .Call("ogrCheckExists", as.character(dsn),
s.character(layer), PACKAGE = "rgdal")
[1] FALSE


As far, as I can see, 'ogrCheckExists' comes from src/ogrsource.cpp.
Something must go wrong in that C++ file, at least for FreeBSD :(

Unfortunalety, I have no skills how to debug such a C++ code within an R
package ...

Any help is really appreciated. Please tell me, if I could try and test
something. My box has FreeBSD, R-devel, gcc-4.8.5, gdb-7.1.0 ...

Greetings,
Rainer Hurling
Roger Bivand
#
On Sun, 8 Nov 2015, Rainer Hurling wrote:

            

      
      
      
    

        
R base::file.exists does include this in its help file: "However, 
directory names must not include a trailing backslash or slash on 
Windows".

so this isn't just rgdal. The dsn= argument for many drivers is a file 
name, not a directory; a work-around seems like overkill.

            

      
      
      
    

        
I do not have access to such a system. As a first step, does 
rgdal::ogrListLayers work correctly?

Beyond that, you'd need to insert Rprintf() statements into the C++ file 
to see whether the FALSE result comes from a failure to open the dsn with 
a known driver, or from the fact that such a layer is not found in that 
dsn.

Does using a different driver help? Does using the shapefile.shp as the 
dsn help (I think this works better in GDAL2)?

Hope this helps,

Roger

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Rainer Hurling
    

    

      
      #
    

  


  

      
Hi Roger,

many thanks for your quick answer.


Am 08.11.15 um 18:02 schrieb Roger Bivand:

            

      
      
      
    

        
OK, that seems reasonable. Of course, rgdal should handle this like R in
total. Thanks you for clarification.

            

      
      
      
    

        
The syntax, shown in the helpfile, does not work for me:

rgdal::ogrListLayers(dsn="/path", layer="layername")
Fehler in rgdal::ogrListLayers(dsn = "/path", layer = "layername") :
  unbenutztes Argument (layer = "layername")    # unused argument


The same, with layer integrated in the dsn, works fine:

rgdal::ogrListLayers(dsn="/path/layername.shp")
[1] "layername"
attr(,"driver")
[1] "ESRI Shapefile"
attr(,"nlayers")
[1] 1


This leads me to the idea, that the same could work for writeOGR or
ogrWrite.R. And yes, this works on FreeBSD:

writeOGR(gis.layer, dsn="/path/layername.shp",
         layer="layername", driver="ESRI Shapefile",
         check_exists=TRUE, overwrite_layer=TRUE)

So, I have to double 'layername' here, once in dsn and once in layer.
But it should be ok as a workaround until I found the real cause.

            

      
      
      
    

        
Thanks for the tip. I will investigate into it.

A first question to this: If I want to integrate some Rprintf into
ogrsource.cpp in the section of ogrCheckExists (lines 832 ...), what is
a good syntax to output for example 'ogrSource' and 'Layer'?

What I tried at ogrsource.cpp:l834 without success is:

Rprintf( "ogrCheckExists: OGRSource %s %p\n", OGRLayer, (void *)
ogrSource) ;

It think, I have to wrap 'ogrSource' with some function here? (I am not
a C/C++ programmer)

            

      
      
      
    

        
Because using the shapefile.shp as the dsn helps, I did not try a
different driver. If this is important for you, I can catch it up later.

Thanks again for your help,
Rainer

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Sun, 8 Nov 2015, Rainer Hurling wrote:

        

            

      
      
      
    

        
The helpfile usage section says:

ogrListLayers(dsn)

which works, as you see below, there is no layer argument, as the 
function queries the dsn to ask what layers it contains (differs between 
drivers).

            

      
      
      
    

        
I have seen this before in GDAL2; it also happened in GDAL1 where the dsn 
directory contained for example *.dbf without *.shp and *.shx. The way dsn 
is handled does vary a lot between drivers.

            

      
      
      
    

        
You'd need simple "No dns found" after line 860 and "No layer found" after 
line 878 to separate the two FALSE returns, but I guess the cause may have 
been in GDAL2, the shapefile driver, and/or what was in your dsn 
directory.

Roger

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    
1 day later

    

      
      

        


  

        

      Rainer Hurling
    

    

      
      #
    

  


  

      
Am 08.11.15 um 20:57 schrieb Roger Bivand:

            

      
      
      
    

        
Ok, as always, I should read documents more careful ;) Thanks for 
clarification again. It seems, the GDAL2 driver for ESRI shapefiles is a 
very special thing ...

            

      
      
      
    

        
I wonder, why it is necessary, to also use the layer="layername" (at 
least on FreeBSD with GDAL2), if the full path and filename in 
dsn="/path/layername.shp" is already given.

For now, I will use this dsn and layer notation for shapefiles with GDAL2.

            

      
      
      
    

        
Ok, I was used to use print statements in debugging code also to get 
variables content, pointers and so on ...

Again, many many thanks for your quick and helpful comments. I now 
understand the driver dependend requirements of GDAL2 a lot more :)

Greetings from G?ttingen,
Rainer

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Tue, 10 Nov 2015, Rainer Hurling wrote:

        

            

      
      
      
    

        
All drivers may be expected to differ in their treatments of the dsn= and 
layer= arguments - both within drivers between GDAL releases and between 
drivers. We benefit from GDAL, and are exposed to the choices changes that 
occur there. Things can be made too "easy" ...

            

      
      
      
    

        
Many of the same types of things happened in < GDAL 2; this version is 
intended to try to be more systematic. Drivers are (were) contributed by 
developers outside GDAL (so not Even Roualt), and change sometimes 
rapidly.

Roger