ggwr and memory problems
On Mon, 21 Jan 2008, Luca Moiana wrote:
Date: Mon, 21 Jan 2008 14:38:18 +0100 From: Roger.Bivand at nhh.no To: luca_moiana at hotmail.com CC: r-sig-geo at stat.math.ethz.ch Subject: Re: [R-sig-Geo] ggwr and memory problems On Mon, 21 Jan 2008, Luca Moiana wrote:
Dear List, Here is my problem: I wanna run a ggwr on a 9000 records Spatial Points Data Frame using R on a Windows Machine (Dual processor, 4 GB RAM).
Have you tuned Windows memory use as discussed in the R for Windows FAQ - section 2.9? The binaries are 32-bit, and need to be told how much memory to use when trying to carry out memory intensive work.
We tried this but didn't change anything.
OK. It may run on Linux, because the memory allocation there accepts many small free patches but Windows wants a single free chunk the size of the request.
When I try to calculate bandwidth using: Sdati14400test.sel <- ggwr.sel(E14400 ~ V211 + V213 + V240 + V313 + V321 + V322 + V331511 + LnMPI25l.max + B:A, family = poisson(link = log), data = Sdati14400test, coords=Sdati14400test.coords, adapt = FALSE, gweight = gwr.gauss, verbose = TRUE, longlat = FALSE) I get a memory allocation error saying that the software is not able to allocate a 749 Mb memory. Any suggestion??
It isn't strictly necessary to use all the observations to find the bandwidth - take a couple of 5% samples and see if the results differ much.
I didn't know that and I would try, but then I'll have memory problems when I try to run ggwr?? Is there a command to obtain a random 5% sample??
Try subsetting the data= argument object: df[o,] with the output of o <- sample(). Remember to say set.seed(whatever) to be able to repeat if need be.
I can also switch and use the same machine with a 64bit Ubuntu SO.
You can try that, but consider dividing the fit.points up into chunks, and running several R processes when actually fitting the ggwr model. The data points stay the same, but fit subsets of the fit.points in separate processes.
I don't have fit.points cause I'm working on the entire Lombardy Region (Northern Italy) and I'd like to compare the model from ggwr with glm models a colleague obtained from a regular glm.
If no fit.points are given, the data points are copied across as fit points internally. You are free to subset the data.points into many fit.points, and concatenate the output objects afterwards. This should remove the difficulty. Roger
MANY THANKS
ggwr() has not (yet) been adapted for using a cluster, but gwr() has and a snow socket cluster will run happily on Linux there, and since it is run within the function, it concatenates the results before returning. If this would be useful of ggwr(), consider taking a look at the code. Roger
THANK A LOT Luca Moiana
_________________________________________________________________ [[alternative HTML version deleted]] _______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no
_________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no