Array indices for very large multi-band images

Dears,

I just joined the list, so please bear with me if my problem
has been dealt with already. I did an extensive google search,
making sure r-gis-geo was included, so am sort of sure my
problem is relevant.

I have a (very) large 3 band ENVI file (32431 x 47262, 4.5 Gb,
i.e. too big for normal GeoTIFF). I would like to generate
statistics for 200 by 200 sized cells (including cross-correlation
between bands).
I am using the rgdal package, trying to use the array indexing
syntax of R, running into trouble when trying to subset all
3 bands at the same time.

Using array index syntax is useful, because it produces a
SpatialGridDataFrame, which makes further processing easier.

Here's my trial script:

require(rgdal)
x<-GDAL.open("verylarge3channelENVIimage")
r <- getRasterData(x, offset = c(14000, 14000), region.dim = c(200, 200))
str(r)
# This works, but produces a simple integer array
int [1:200, 1:200, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
# I'd rather do this
sub0 <- x[14000:14199, 14000:14199,] ?# ... but doesn't work
Error in x[14000:14199, 14000:14199, ] :
?argument is missing, with no default
# this then?
sub1 <- x[14000:14199, 14000:14199,1:3] # no, running into memory trouble
Error: cannot allocate vector of size 2.3 Gb
# per band works:
sub2 <- x[14000:14199, 14000:14199,1] ?# produces a nice spatial object.
summary(sub2)
Object of class SpatialGridDataFrame
...

Is there a way to get all 3 bands with array index syntax, preferably
as a SpatialGridDataFrame with band1, band2, band3.

Any help greatly appreciated,

GL

Error: cannot allocate vector of size 2.3 Gb

Guido,

Perhaps the function getValuesBlock in 'raster' makes this easier to do:

library(raster)
b <- brick(system.file("external/Rlogo.grd", package="raster"))
getValuesBlock(b, row=1, nrows=3, col=1, ncols=3)

Robert

On Fri, Apr 9, 2010 at 3:05 AM,  <guido.lemoine at jrc.ec.europa.eu> wrote:

Dears,

I just joined the list, so please bear with me if my problem
has been dealt with already. I did an extensive google search,
making sure r-gis-geo was included, so am sort of sure my
problem is relevant.

I have a (very) large 3 band ENVI file (32431 x 47262, 4.5 Gb,
i.e. too big for normal GeoTIFF). I would like to generate
statistics for 200 by 200 sized cells (including cross-correlation
between bands).
I am using the rgdal package, trying to use the array indexing
syntax of R, running into trouble when trying to subset all
3 bands at the same time.

Using array index syntax is useful, because it produces a
SpatialGridDataFrame, which makes further processing easier.

Here's my trial script:

require(rgdal)
x<-GDAL.open("verylarge3channelENVIimage")
r <- getRasterData(x, offset = c(14000, 14000), region.dim = c(200, 200))
str(r)
# This works, but produces a simple integer array
int [1:200, 1:200, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
# I'd rather do this
sub0 <- x[14000:14199, 14000:14199,]  # ... but doesn't work
Error in x[14000:14199, 14000:14199, ] :
 argument is missing, with no default
# this then?
sub1 <- x[14000:14199, 14000:14199,1:3] # no, running into memory trouble
Error: cannot allocate vector of size 2.3 Gb
# per band works:
sub2 <- x[14000:14199, 14000:14199,1]  # produces a nice spatial object.
summary(sub2)
Object of class SpatialGridDataFrame
...

Is there a way to get all 3 bands with array index syntax, preferably
as a SpatialGridDataFrame with band1, band2, band3.

Any help greatly appreciated,

GL