reading ascii grid memory issues

Thanks Barry

Hi List,

I am unable to read in a 7.8Gb ascii grid using readAsciiGrid {maptools}/ read.asciigrid {sp}; R runs out of memory. I have 4Gb ram and 4Gb swap, so things are getting tight.

I am wondering if anyone has alternative options (preferably with example) that enable to read in large grids, do some calculations and save a new grid?

Depends on what you want to do with it. What's the dimension of the grid in rows x columns?

-->dimensions 80264x73772 for the largest grid and 28848x47030 for current grid

Firstly, can you convert it to a more efficient format? Ascii Grid files are generally long lines of numbers, looking like 2.333535
4.457574 2.332336 5.445775 and so on. Maybe 10 characters per grid square. You might be able to squeeze it to 4 characters per grid square by storing as raw floating point numbers.

-->yes, depends on what formats R is able to read in

 Next, can you just process it serially, one line or grid square at a time? Do you need to store it all in memory at the same time?

--> I can process serially, I don't need to store it in memory for the simple math calculations, only for subsequent analysis. But once calculations are done I can resample/aggregate to higher resolution.