An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20130718/0857f249/attachment.pl>
working with large ascii files in R
5 messages · Rocio Ponce, Roman Luštrik, Mauricio Zambrano-Bigiarini +1 more
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20130718/c41b2705/attachment.pl>
On 07/18/2013 09:26 AM, Roman Lu?trik wrote:
Have you considered updating your R and associated packages?
As Roman mentioned, I also suggest you, as first step, to update R (and
all the packages) up to version 3 or higher, because one of the
significant user changes introduced in R 3.0.0 was related to memory
(from the NEWS file):
"It is now possible for 64-bit builds to allocate amounts of
memory limited only by the OS. It may be wise to use OS
facilities (e.g. ulimit in a bash shell, limit in csh), to set
limits on overall memory consumption of an R process,
particularly in a multi-user environment. A number of packages
need a limit of at least 4GB of virtual memory to load.
64-bit Windows builds of R are by default limited in memory usage
to the amount of RAM installed: this limit can be changed by
command-line option --max-mem-size or setting environment
variable R_MAX_MEM_SIZE."
Kind regards,
Mauricio Zambrano-Bigiarini, Ph.D
================================================= Water Resources Unit Institute for Environment and Sustainability (IES) Joint Research Centre (JRC), European Commission TP 261, Via Enrico Fermi 2749, 21027 Ispra (VA), IT Work Phone : +39 0332 789588 webinfo : http://floods.jrc.ec.europa.eu/ ================================================= DISCLAIMER: "The views expressed are purely those of the writer and may not in any circumstances be regarded as sta- ting an official position of the European Commission" ================================================= "The journey is the reward" (Steve Jobs) > There's also a plethora of other GUI based GIS systems. Just the other day, > I found Quantum GIS to be very user friendly. > > That being said, I suggest you use R. :) > > Cheers, > Roman > > > > On Thu, Jul 18, 2013 at 9:05 AM, Rocio Ponce <r.ponce.reyes at gmail.com>wrote: > >> Hi there, >> I'm a newie in using R to make maps... I used to work with ArcGis but I ran >> out of a valid license... >> >> I'm trying to crop the future Worldclim datasets layers which are huge >> ascii files (of the whole world at 1km resolution, so they are between >> 3-4GB each) to the extent of Mexico using the library (raster). >> >> I used the following code: >> >>> setwd('G:/future_A2a/2020/bccr_bcm2_0_sres_a2_2020s/') >>> maask<- read.asc("C:/xalapa/worldclim_pres/bio01.asc",gz=FALSE) >>> mask<-raster.from.asc(maask,projs=NA) >>> files <- list.files(pattern='\\.asc$') >>> s <- stack(files) >>> crop(s, mask) >> Error: cannot allocate vector of size 253.2 Mb >> >> >> I'm using R.2.14.1 on a computer running on Windows XP >> Any help that you could provide me would be greatly appreciated, >> >> Cheers, >> Rocio >> >> -- >> Dr. Rocio Ponce-Reyes >> >> http://rocioponcereyes.wordpress.com/ >> >> (disculpe la falta de acentos en este correo electronico) >> >> [[alternative HTML version deleted]] >> >> _______________________________________________ >> R-sig-Geo mailing list >> R-sig-Geo at r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > > >
Rocio,
You should not use functions read.asc and raster.from.asc because they
read all the values into memory. Also please indicated which packages
you use. The below should fix the problem:
setwd('G:/future_A2a/2020/bccr_bcm2_0_sres_a2_2020s/')
library(raster)
mask<-raster("C:/xalapa/worldclim_pres/bio01.asc")
files <- list.files(pattern='\\.asc$')
s <- stack(files)
x <- crop(s, mask)
And it would indeed good to use a current version of R and its packages.
Robert
On Thu, Jul 18, 2013 at 12:05 AM, Rocio Ponce <r.ponce.reyes at gmail.com> wrote:
Hi there, I'm a newie in using R to make maps... I used to work with ArcGis but I ran out of a valid license... I'm trying to crop the future Worldclim datasets layers which are huge ascii files (of the whole world at 1km resolution, so they are between 3-4GB each) to the extent of Mexico using the library (raster). I used the following code:
setwd('G:/future_A2a/2020/bccr_bcm2_0_sres_a2_2020s/')
maask<- read.asc("C:/xalapa/worldclim_pres/bio01.asc",gz=FALSE)
mask<-raster.from.asc(maask,projs=NA)
files <- list.files(pattern='\\.asc$')
s <- stack(files)
crop(s, mask)
Error: cannot allocate vector of size 253.2 Mb I'm using R.2.14.1 on a computer running on Windows XP Any help that you could provide me would be greatly appreciated, Cheers, Rocio -- Dr. Rocio Ponce-Reyes http://rocioponcereyes.wordpress.com/ (disculpe la falta de acentos en este correo electronico) [[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20130718/312e3a1c/attachment.pl>