Making web-repository of gridded maps: NetCDF or WKT Raster?
Dear Edzer, Barry, Servet, and Miguel, Thank you for your tips and for sharing your experiences with us (R-sig-geo keeps on surprising me considering the amount of support I get :). I will dig into the links you have forwarded. Most probably, we will try to implement two/three paths and then compare them to see which is 'better' (the main issue is that we want to do majority of processing on the server): (A) The rasdaman approach (http://kahlua.eecs.jacobs-university.de/trac/rasdaman/wiki/Rasdaman_8.0) 1. Install PostgreSQL support on server and download and install rasdaman File Set; 2. Create a database and put the grids in PostgreSQL database (raslib applications); 3. Test some examples with overlaying, subsetting, resampling, upscaling and visualizing data (WMS, KML); (B) The OPeNDAP/NetCDF approach (http://www.unidata.ucar.edu/software/netcdf/software.html#GDAL) 1. Install OPeNDAP server and UMN Mapserver to serve data; 2. Put all grids into a single (CF)NetCDF file and upload (using WebDAV) to the server; 3. Test some examples (R + GDAL) with overlaying, subsetting, resampling, upscaling and visualizing the data (WMS, KML); I will keep you informed once we produce some live examples. We might first start testing a WCPS with the worldmaps repository (http://spatial-analyst.net/worldmaps/); then develop some examples (R code) and test flexibility/speed of both approaches. cheers, T. Hengl http://home.medewerker.uva.nl/t.hengl/
-----Original Message----- From: r-sig-geo-bounces at stat.math.ethz.ch [mailto:r-sig-geo-bounces at stat.math.ethz.ch] On Behalf Of Edzer J. Pebesma Sent: Sunday, December 13, 2009 9:05 PM To: Stefano Casalegno Cc: r-sig-geo Subject: Re: [R-sig-Geo] Making web-repository of gridded maps: NetCDF or WKT Raster? Dear Tom, as your grids are spatial grids, you may want to look into the web service standards of the OGC (www.opengeospatial.org) for them, WCS (for reading) and WCS-T (for writing) in particular. We're experimenting with this currently, using the recently open sourced rasdaman software (www.rasdaman.org), which uses a data base in the back end. I don't think OPeNDAP has special infrastructure for spatial grids; CFNetCDF does, but by choosing this you may limit integration with other software that does assume spatial data to be served over OGC standards. CFNetCDF is one of the possible formats to be served over WCS (and GeoTIFF etc) -- if I'm correctly informed. Bests, -- Edzer Stefano Casalegno wrote:
Dear Tom, my best acknowledgment for the Ecogrid challenging initiative. I guess the main limitation will be "time" and this is related to the size of grids. What is the extent of the grids ? What tool you plan to use for point 1. Overlay some point data and get the values of grids ? On a more theoretical basis, "distribution models should rely upon data that covers the whole extent of the distribution of the biota under examination" Please consider forest data from http://forest.jrc.ec.europa.eu/climate-change as supplementary predictors variables within an European biodiversity and nature conservation application. Cheers Stefano
________________________________ Stefano Casalegno, Ph.D. via Greppi 16, 21021 Angera - Italy email: stefano at casalegno.net web: http://www.casalegno.net/stefano ________________________________ On Dec 13, 2009, at 6:29 PM, Tomislav Hengl wrote: Dear R-sig-geo, As a part of our project (EcoGRID.nl) we have prepared some 60 thematic grids that we use as auxiliary predictors for species distribution modeling. At this stage, we would like to put the gridded maps (50/100 m base resolution) into some efficient sharable geo-database. We will most probably put the data into the NetCDF format (http:// www.unidata.ucar.edu/software/netcdf/) because it can handle any- dimensional array data, and because it has been in continuous development and widespread use for many years. NetCDF grids can be read relatively easy into R using the RNetCDF package (e.g. http:// spatial-analyst.net/DATA/readNCDF.zip). Another alternative is to use PostGIS WKT Raster format (http://trac.osgeo.org/postgis/wiki/ WKTRaster), but this seems to be still rather experimental (?). Once we put the grids into NetCDF format. We plan to install OPeNDAP (http://www.opendap.org) server on top to make the files accessible through the web; then Geoserver (http:// www.geoserver.org) or UMN Mapserver (http://mapserver.org) to feed a WMS from NetCDF files (raster data). Finally, we plan to add a simple OpenLayers interface on top of that (Geoserver has it built in) to allow direct browsing of the data and metadata (e.g. such as this one: http://africamap.harvard.edu/) Just to be clear, we want to put the data on a server because we would like to run a number of operations directly on the server (via rgdal?): 1. Overlay some point data and get the values of grids (without a need to download the grids locally); 2. Subset/mask and resample gridded data of interest (for a given bounding box and proj4 string; again without a need to download the data locally); 3. Upscale the grids from 100 m to 250, 500 m and 1 km resolution (then download the upscaled grids). 4. Write/upload new grids to the database (e.g. using WebDAV). 5. Browse the grids (and metadata) via the OpenLayers. These are only our wishes of course. We do not know if all this is really possible with the current software. Any examples or comments/suggestions/experiences are welcome (before we start installing and testing the functionality). Thanx! Tomislav Hengl and Lourens Veen _______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo [[alternative HTML version deleted]] _______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo