Large dataset

Hi all, I have a large dataset of 360,000 sample points.  When I perform
analysis using Khat function of splancs, the following error messages
prompted:

Error: cannot allocate vector of size 589.6 Mb
In addition: Warning messages:
1: Reached total allocation of 1013Mb: see help(memory.size)

It would be very much appreciated if someone could advice me how to
overcome this problem.

The obvious first question is why? If you take a sample of 10000 points, 
you will get a very good approximation of the khat value at the s-bins - 
so why try all 360K? Either take tiles of your data set, or some other 
partition - since you are assuming anyway that the process is the same 
over the whole area, aren't you? You will certainly find that simulating 
from a CSR for 360K to put an envelope on the result will also be memory 
intensive.

By the way, always include your specific code, and the output of 
sessionInfo(). It may be that you have a very complex polygon windoe, 
and/or too much detail in the s-bins. Do you get the same memory 
constraint in spatstat, or equivalently in spatial?

I assume that you are looking to see if the point pattern is clustered, is 
that correct? If you really need to use all the points, we'd need to debug 
the command to see where the memory allocation is occurring, since it may 
be an avoidable copying operation somewhere. More RAM is another option.

Roger

Dr Kam Tin Seong
Practice Associate Professor
School of Information Systems

Singapore Management University
80 Stamford Road
Singapore 178902
e-mail: tskam at smu.edu.sg
DID: + 65 6828 0932



	[[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at stat.math.ethz.ch
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no