Memory issue
On Mon, 3 Nov 2008, Mikael Carlsson wrote:
Hi,
?
First, thanks for all help so far...but I still have some problems.
Here is the code:
cumark <- read.table("k:/test/helamarkcu.txt", header=T, sep = ";")
One copy of 3*40K*8 = 1MB
memory.limit(size = 1897)??? attach(cumark) vew <- as.vector(ew) vns <- as.vector(ns) vcu <- as.vector(cu)
A second copy
fcumark <- data.frame(vew, vns, vcu)
A third copy (but still of 1MB)
coordinates(fcumark) <- ~vew+vns? L1 <-round((max(vew)- min(vew))/100,0) L2 <-round((max(vns)- min(vns))/100,0)
So this is a 100m grid? What are L1 and L2? Why not use GridTopology to compute this? How many grid cells are you using?
fcumark.interp.old <- interp.old(vew, vns, vcu, ??????????????? xo= seq(min(vew), max(vew), length = L1), ??????????????? yo= seq(min(vns), max(vns), length = L2), ??????????????? ncp = 0,? ??????????????? extrap= FALSE, ??????????????? duplicate = "mean", ??????????????? dupfun = NULL) Then I get this; Error: cannot allocate vector of size 351.8 Mb When I tested with a smaller file (~2000 rows) I come to the end without problems.
With the same L1, L2 (possibly not)? They must be the main problem, no? As suggested, you could tile the interpolations and merge them afterwards. The main thing is the size of L1*L2.
Rest of the code.. fcumark.interp.old.grid <- as.SpatialGridDataFrame.im(as.im(fcumark.interp.old))
Lots more copying
##? spplot(fcumark.interp.old.grid, col.regions=bpy.colors(),
sp.layout=list("sp.points", pch="+", fcumark))
slot(slot(fcumark.interp.old.grid, "grid"), "cellsize") <-
rep(mean(slot(slot(fcumark.interp.old.grid,
?? "grid"), "cellsize")), 2)
Messy - use a Grid Topology as input. Roger
write.asciigrid(fcumark.interp.old.grid[1], "k:/test/helamarkcu.asc", na.value = -9999) ## End run helamarkcu.txt is a plain ascii file, looks like... ew;ns;cu 1344628;6138662;24.0 1340104;6138880;17.0 1342246;6138946;21.0 1347259;6140911;19.0 1339968;6140932;15.0 1349906;6140987;21.0 ... .. . (38155 rows) Have anybody a solution? Writing result from interp.old direct to disk/file on the "fly"? Running R 2.7.2?and WinXP Regards, Mikael
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no