Computing very large distance matrix
On Wed, 25 Feb 2004, Arnav Sheth wrote:
Hello All,
I have a 131072x132 matrix for which I need to compute a regular euclidean distance matrix, which I then need to transform and run agnes() on this transformed matrix. I am having trouble computing the distance matrix as it is fairly large and I am sure I have gone over the max.
The specific error I am getting is:
Error in vector("double", length) : negative length vectors are not allowed
We've improved that message for the next release.
I have increased the memory limit to the maximum capacity of my hard drive (which is around 20gb), with no success.
You'd need to increase the 32-bit limit of Windows, too ....
I am running the RGUI on Windows XP with 512 mb of RAM. Would anyone have any suggestions as to how I can overcome this problem? I would be most grateful for any help.
No. The distance matrix is 8Gb all by itself.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595