decimals in NA value using raster::writeRaster with .ascii format
On 04/11/2014 05:46 PM, Zack Holden wrote:
Robert, Thank you for your help. I failed to notice the "datatype vs. dataType difference. Calling 'datatype' does indeed solve my integer test case. However, some of the raster datasets that I am processing are actually floating point. When I call writeRaster(datatype = 'FLT4S'), I still see the long string of decimals in the first value in my ascii grid. Is there any way to handle this in R? From your response, it sounds like this might be an artifact of GDAL, and I may need to manually (or with text manipulations) edit out the decimal strings.
Look for https://r-forge.r-project.org/scm/viewvc.php/pkg/raster/R/writeAllAscii.R?view=markup&root=raster and lower the 15, or else change the formatC call, then rebuild raster. As described earlier, raster does this on purpose: the gdal asciigrid reader makes the wrong assumption about the data type when a grid starts with too many missing values that look like an integer (like -9999 but unlike -9999.0). This has been reported over a decade ago, and may be seen as an indication of gdal's liveliness. Not only OpenSSL suffers from too few eyes to make bugs shallow. What software are you using that can't digest the 15-digit float?
Thanks for any additional suggestions. Javier, thanks for pointing out the available GDAL drivers in PCraster, I'm just getting acquainted with that software program. Cheers, Zack On Fri, Apr 11, 2014 at 8:48 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
Robert: What is the practical difference between those two? -- does dataType get passed off to GDAL, while datatype is interpreted by your R code? --j On Thu, Apr 10, 2014 at 7:06 PM, Robert J. Hijmans <r.hijmans at gmail.com> wrote:
Zack, You are using: dataType="INT2U" but it should be: datatype="INT2U" # (or any other INT data type) That fixes it. If the datatype is FLT* (the default), the first value gets decimals to assure that GDAL and others treat the values as floats, not integers. Robert On Thu, Apr 10, 2014 at 3:36 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
Zack: Hopefully Robert will get back to you (I confirmed it -- it looks like a bug to me) -- but in the meantime, you might want to try my gdalUtils package, which uses gdal_translate to convert files. You can use one of the many ASCII formats -- you'll want to use parameters like: gdal_translate(...,a_nodata=-9999,ot="UInt16",format="...") You can see the available formats at: http://www.gdal.org/formats_list.html --j -- On Thu, Apr 10, 2014 at 5:29 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
Thanks for getting back to me Jonathan. In reality, I am creating the files from an existing DEM. However, I get the same behavior regardless of whether I generate the file in R (as with my example code) or with actual data. I have since tested on 4 machines. One windows machine with an older version of R successfully generates the desired ascii file without the decimals. The other 3 computers (Linux and windows machines) add the decimals. As a workaround, I've tried using -Inf as my NAflag. This eliminates the decimals, but results in a large string of white space in front of the first row of data. These still cause problems for PCraster (the next step in the processing chain). I'm still stumped. Thanks again, Zack On Thu, Apr 10, 2014 at 4:22 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
Zack: Are you creating the files from within R, or are you just trying to convert an existing file? If the latter, can you make one of those files available (e.g. via google drive or dropbox or something like that)? --j On Thu, Apr 10, 2014 at 3:06 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
Dear list,
I am creating ascii files as inputs to the PCraster software (no other
gdal format is allowed). When I write my ascii file using
raster::writeRaster, the first cell of every raster that I write has a
large number of decimal values appended to it. This causes errors
later in the model workflow.
This also happens to the first NA value in the file, so that all NA
values in the file are e.g. -9999 except the first cell. which appears
as: -9999.00000000000
Can anyone suggest a method for ensuring that all of my NA values are
identical and with no trailing decimals?
A small reproducible example follows. Any help or suggestions is appreciated.
Thanks,
Zack
##################
# start code
require(raster)
r <- raster(nrows=2, ncols=2, xmn=0, xmx=2, ymn=0, ymx=2)
r <- setValues(r, seq(1:ncell(r)))
rexpand <- extend(r, c(2,2))
NAvalue(rexpand) <- -9999
temp.dir <- "/home/zholden/"
writeRaster(rexpand, file=paste(temp.dir, "testraster.asc",sep=""),
NAflag=-9999, dataType="INT2U", overwrite=T)
# end code
##################
# opening in a text editor reveals trailing decimals in the first NA value.
system(paste("gedit ", temp.dir, "testraster.asc", sep=""))
################################
sessionInfo()
R version 3.0.2 (2013-09-25)
Platform: x86_64-redhat-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] rgdal_0.8-16 raster_2.2-12 sp_1.0-14
loaded via a namespace (and not attached):
[1] grid_3.0.2 lattice_0.20-23 tools_3.0.2
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
-- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Edzer Pebesma Institute for Geoinformatics (ifgi), University of M?nster Heisenbergstra?e 2, 48149 M?nster, Germany. Phone: +49 251 83 33081 http://ifgi.uni-muenster.de GPG key ID 0xAC227795 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: OpenPGP digital signature URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20140414/603c28f5/attachment.bin>