memory problems with GWR tests
On Mon, 19 May 2008, Michal Gallay wrote:
Dear R-Experts, there is an error message regarding allocation of memory coming up after attempting to run GWR with hatmatrix=TRUE (there is no problem when this is set to FALSE). I would like to test GWR outputs using e.g. LMZ.F2GWR.test(x)or BFC99.gwr.test(x).
resid.Gauss <- gwr(regress.formula, data=resid, gweight=gwr.Gauss, bandwidth=10, hatmatrix=TRUE))
Error: cannot allocate vector of size 505.4 Mb Why it cannot allocate 505 MB if there is 2 GB RAM on my laptop? Running R from WinXP command prompt: Rgui --max-mem-size=1800Mb The memory problem persists. I read the instructions regarding the memory.limit(), Memory(), as well as the R help.I could decimate the number of points involved in GWR but that would be a test on a different scale, wouldn't it? The datasets comprise 8 000 - 1 000 000 points. A Monte Carlo simultation would require too much time. I would like to know if there would be any way around the memory problem to test a GWR object.
Absolutely no way round at all, the hat matrix must be n x n (and you need several copies typically). In addition, the tests only have meaning if the assumptions they require are met, which under non-stationarity they typically are not. So if you need to test, choose several subsets of observations, and see if the test results correspond. Hope this helps, Roger
I much appreciate any advice. Michal PS: There is some other information about the session: My seesion info: sessionInfo() R version 2.6.2 (2008-02-08) i386-pc-mingw32 locale: LC_COLLATE=Slovak_Slovakia.1250;LC_CTYPE=Slovak_Slovakia.1250;LC_MONETARY=Slovak_Slovakia.1250;LC_NUMERIC=C;LC_TIME=Slovak_Slovakia.1250 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] lattice_0.17-6 spgwr_0.5-1 maptools_0.6-5 foreign_0.8-23 sp_0.9-23 loaded via a namespace (and not attached): [1] grid_2.6.2
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 269045 7.2 531268 14.2 531268 14.2 Vcells 413865 3.2 84325970 643.4 105241855 803.0
memory.profile()
NULL symbol pairlist closure environment promise
1 6876 143458 3669 825 5271
language special builtin char logical integer
37538 151 1464 16489 4324 5293 double
complex character ... any list
2446 1 36010 0 0 3262
expression bytecode externalptr weakref raw
1 0 665 177 1
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no