Making web-repository of gridded maps: NetCDF or WKT Raster?
-----Original Message----- From: Pierre Racine [mailto:Pierre.Racine at sbf.ulaval.ca] Sent: Friday, February 05, 2010 4:49 PM To: Tomislav Hengl; r-sig-geo Subject: RE: [R-sig-Geo] Making web-repository of gridded maps: NetCDF or WKT Raster? Hi Tom, I understand that PostGIS WKT Raster is still under development but it is designed to answer exactly the kind of problem you are confronted with: serving as a warehouse for huge raster datasets, integrate them with vector (PostGIS) through raster/vector GIS operations (raster/vector overlay, raster operations like resampling, mask, mapalgebra), all this using a common SQL interface. WKT Raster intent to be a final elegant solution to this kind of problem avoiding the development of multi layer applications like the one you are envisioning. WKT Raster intend to make of PostGIS a complete SQL GIS solution handling both vector and raster data. As a matter of example we started the WKT Raster project to be able to intersect thousands of vector buffers with many raster dataset covering the extent of Canada (many in the 30 GIG order). Our final goal is to enable our users to do that on the web (even if some query may take days). Note however that WKT Raster is not a web server. MapServer, Geoserver and OpenLayer will have to be adapted to support WKT Raster. There is already a GDAL driver able to read WKT Raster from the database. This said, I invite you to consider a contribution to the WKT Raster project (or one to the other web project like MapServer, GeoServer or OpenLayer: adding WKT Raster support to them) in order to concentrate your efforts on a generic solution to problems similar to your one, contributing to the whole community, instead of disseminating your energy on a specific solution, serving only your organisation.
Dear Pierre, I am sorry that you might got me wrong (and thank you for your reply even with some months of delay!). I do intend to test also WKT Raster and will certainly report on my results via R-sig-geo. We are just in a process of setting up our server and installing all functionality. Provided that we find your software suitable, efficient and secure (in comparison to e.g. rasdaman or NetCDF), I would be willing to contribute more actively e.g. to your wiki [http://trac.osgeo.org/postgis/wiki/WKTRaster] (certainly by reporting on the results using same sample data-sets). I am not sure about what kind of set-up do you have with your employer, but my project (EcoGRID, client: Dutch Ministry of Agriculture, Nature and Food Quality) needs to stay a priority of course. The art is to be able to satisfy your employer and still contribute to open access projects such as yours. T. Hengl http://home.medewerker.uva.nl/t.hengl/
Thanks, Pierre Racine WKT Raster Project Manager
-----Original Message----- From: r-sig-geo-bounces at stat.math.ethz.ch [mailto:r-sig-geo-
bounces at stat.math.ethz.ch] On Behalf Of
Tomislav Hengl Sent: 13 d?cembre 2009 12:29 To: r-sig-geo Subject: [R-sig-Geo] Making web-repository of gridded maps: NetCDF or WKT
Raster?
Dear R-sig-geo, As a part of our project (EcoGRID.nl) we have prepared some 60 thematic grids that we use as auxiliary predictors for species distribution modeling. At this stage, we would like to put the gridded maps (50/100 m base resolution) into some efficient sharable geo-database. We will most probably put the data into the NetCDF format (http://www.unidata.ucar.edu/software/netcdf/) because it can handle any-dimensional array data, and because it has been in continuous development and widespread use for many years. NetCDF grids can be read relatively easy into R using the RNetCDF package (e.g. http://spatial-analyst.net/DATA/readNCDF.zip). Another alternative is to use PostGIS WKT Raster format (http://trac.osgeo.org/postgis/wiki/WKTRaster), but this seems to be still rather experimental (?). Once we put the grids into NetCDF format. We plan to install OPeNDAP (http://www.opendap.org) server on top to make the files accessible through the web; then Geoserver (http://www.geoserver.org) or UMN Mapserver (http://mapserver.org) to feed a WMS from NetCDF files (raster data). Finally, we plan to add a simple OpenLayers interface on top of that (Geoserver has it built in) to allow direct browsing of the data and metadata (e.g. such as this one: http://africamap.harvard.edu/) Just to be clear, we want to put the data on a server because we would like to run a number of operations directly on the server (via rgdal?): 1. Overlay some point data and get the values of grids (without a need to download the grids locally); 2. Subset/mask and resample gridded data of interest (for a given bounding box and proj4 string; again without a need to download the data locally); 3. Upscale the grids from 100 m to 250, 500 m and 1 km resolution (then download the upscaled grids). 4. Write/upload new grids to the database (e.g. using WebDAV). 5. Browse the grids (and metadata) via the OpenLayers. These are only our wishes of course. We do not know if all this is really possible with the current software. Any examples or comments/suggestions/experiences are welcome (before we start installing and testing the functionality). Thanx! Tomislav Hengl and Lourens Veen
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo