An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20130306/1c16ab80/attachment.pl>
readGDAL re-mapping data to 0-255
5 messages · Michael Sumner, Biuw, Martin, Roger Bivand
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20130306/72175629/attachment.pl>
On Wed, 6 Mar 2013, Biuw, Martin wrote:
Hi, I'm using readGDAL to read data from a geoTiff file. The valid data range of the single band is -26 to 0. I have two versions of the file, one with a NoDataValue = -9999 and one with NoDataValue = -128. When reading the version with -9999 everything works great, and I get the data values in band1 mapped correctly on the original interval. But when I use the version of the file with NoDataValue = -128, these band1 data values get re-mapped to 0-255.
This appears to indicate that the files do not only differ in NoDataValue, but also in storage form. Does examining the output of GDALinfo() on both tell you anything? Is one a signed 16-bit integer, the other Byte? Maybe paste some of the output into a follow-up? Roger
I remember reading a helpfile or R-sig post on why the NoDataValue has such an effect, or what can be done to get around it. But can't for the life of me find that message anymore. The reason I need to use the file with NoDataValue = -128 is that this is in UTM 35 while the other one is in UTM 33. Since it's a SpatialGridDataFrames, I'd rather not use spTransform on it, since this turns it into a SpatialPointsDataFrame. The files were originally created (by someone else) in ArcGIS, so I could always ask them to re-create them with correct NoDataValue. But maybe there's a way to explicitly tell readGDAL how to treat the data values? I've desperately been looking for any help on this, but must be looking in the wrong place.... Thanks in advance, Martin
________________________________________________________________ Martin Biuw (PhD) Researcher Norwegian Institute for Nature Research - NINA Postal address: Framsenteret, 9296 Troms?, NORWAY Delivery/Visiting address: Framsenteret, Hjalmar Johansens gate 14, 9007 Troms?, NORWAY Phone: +47 77 75 04 17 * Cell: +47 40 72 96 15 * Fax: +47 77 75 04 01 * www.nina.no<http://www.nina.no/> ________________________________________________________________ [[alternative HTML version deleted]]
Roger Bivand Department of Economics, NHH Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no
Ah, that was the problem.
The file with -9999 that works fine is Int16:
apparent band summary:
GDType Bmin Bmax Bmean Bsd hasNoDataValue NoDataValue
1 Int16 -26 0 -3.130923 3.832138 TRUE -9999
While the other one is Byte:
apparent band summary:
GDType Bmin Bmax Bmean Bsd hasNoDataValue NoDataValue
1 Byte -26 0 -3.089263 3.864005 TRUE -128
So this sort of problem basically stems from the way ArcGIS saves the geoTiff depending on the data range? Or is there a way to specify to readGDAL how to deal with Byte or Int16 data?
Thanks for the help!
Martin
-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
Sent: 06 March 2013 12:47
To: Biuw, Martin
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] readGDAL re-mapping data to 0-255
On Wed, 6 Mar 2013, Biuw, Martin wrote:
Hi, I'm using readGDAL to read data from a geoTiff file. The valid data range of the single band is -26 to 0. I have two versions of the file, one with a NoDataValue = -9999 and one with NoDataValue = -128. When reading the version with -9999 everything works great, and I get the data values in band1 mapped correctly on the original interval. But when I use the version of the file with NoDataValue = -128, these band1 data values get re-mapped to 0-255.
This appears to indicate that the files do not only differ in NoDataValue, but also in storage form. Does examining the output of GDALinfo() on both tell you anything? Is one a signed 16-bit integer, the other Byte? Maybe paste some of the output into a follow-up? Roger
I remember reading a helpfile or R-sig post on why the NoDataValue has such an effect, or what can be done to get around it. But can't for the life of me find that message anymore. The reason I need to use the file with NoDataValue = -128 is that this is in UTM 35 while the other one is in UTM 33. Since it's a SpatialGridDataFrames, I'd rather not use spTransform on it, since this turns it into a SpatialPointsDataFrame. The files were originally created (by someone else) in ArcGIS, so I could always ask them to re-create them with correct NoDataValue. But maybe there's a way to explicitly tell readGDAL how to treat the data values? I've desperately been looking for any help on this, but must be looking in the wrong place.... Thanks in advance, Martin
________________________________________________________________ Martin Biuw (PhD) Researcher Norwegian Institute for Nature Research - NINA Postal address: Framsenteret, 9296 Troms?, NORWAY Delivery/Visiting address: Framsenteret, Hjalmar Johansens gate 14, 9007 Troms?, NORWAY Phone: +47 77 75 04 17 * Cell: +47 40 72 96 15 * Fax: +47 77 75 04 01 * www.nina.no<http://www.nina.no/> ________________________________________________________________ [[alternative HTML version deleted]]
-- Roger Bivand Department of Economics, NHH Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no
On Wed, 6 Mar 2013, Biuw, Martin wrote:
Ah, that was the problem.
The file with -9999 that works fine is Int16:
apparent band summary:
GDType Bmin Bmax Bmean Bsd hasNoDataValue NoDataValue
1 Int16 -26 0 -3.130923 3.832138 TRUE -9999
While the other one is Byte:
apparent band summary:
GDType Bmin Bmax Bmean Bsd hasNoDataValue NoDataValue
1 Byte -26 0 -3.089263 3.864005 TRUE -128
So this sort of problem basically stems from the way ArcGIS saves the
geoTiff depending on the data range? Or is there a way to specify to
readGDAL how to deal with Byte or Int16 data?
If the input data are Byte, you may try the rgdal toSigned() function: x <- c(1:10, 127:129, 240:256) toSigned(x, 8) which probably will not respect the -128, but you can then untangle that later with is.na(). You'll need to check the output as the function is not used often and may not do what you need. The function was added to the package in September 2009: https://stat.ethz.ch/pipermail/r-sig-geo/2009-September/006387.html Hope this helps, Roger
Thanks for the help! Martin -----Original Message----- From: Roger Bivand [mailto:Roger.Bivand at nhh.no] Sent: 06 March 2013 12:47 To: Biuw, Martin Cc: r-sig-geo at r-project.org Subject: Re: [R-sig-Geo] readGDAL re-mapping data to 0-255 On Wed, 6 Mar 2013, Biuw, Martin wrote:
Hi, I'm using readGDAL to read data from a geoTiff file. The valid data range of the single band is -26 to 0. I have two versions of the file, one with a NoDataValue = -9999 and one with NoDataValue = -128. When reading the version with -9999 everything works great, and I get the data values in band1 mapped correctly on the original interval. But when I use the version of the file with NoDataValue = -128, these band1 data values get re-mapped to 0-255.
This appears to indicate that the files do not only differ in NoDataValue, but also in storage form. Does examining the output of GDALinfo() on both tell you anything? Is one a signed 16-bit integer, the other Byte? Maybe paste some of the output into a follow-up? Roger
I remember reading a helpfile or R-sig post on why the NoDataValue has such an effect, or what can be done to get around it. But can't for the life of me find that message anymore. The reason I need to use the file with NoDataValue = -128 is that this is in UTM 35 while the other one is in UTM 33. Since it's a SpatialGridDataFrames, I'd rather not use spTransform on it, since this turns it into a SpatialPointsDataFrame. The files were originally created (by someone else) in ArcGIS, so I could always ask them to re-create them with correct NoDataValue. But maybe there's a way to explicitly tell readGDAL how to treat the data values? I've desperately been looking for any help on this, but must be looking in the wrong place.... Thanks in advance, Martin
________________________________________________________________ Martin Biuw (PhD) Researcher Norwegian Institute for Nature Research - NINA Postal address: Framsenteret, 9296 Troms?, NORWAY Delivery/Visiting address: Framsenteret, Hjalmar Johansens gate 14, 9007 Troms?, NORWAY Phone: +47 77 75 04 17 * Cell: +47 40 72 96 15 * Fax: +47 77 75 04 01 * www.nina.no<http://www.nina.no/> ________________________________________________________________ [[alternative HTML version deleted]]
-- Roger Bivand Department of Economics, NHH Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no
Roger Bivand Department of Economics, NHH Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no