Computing very large distance matrix

Hello,

Thank you all for your replies.

I have just discovered that now, for some reason, R does not let me increase
its memory limit beyond 4095 Mb.

Also, I get a different error message when computing the matrix using
daisy() as opposed to dist():

Error: Cannot allocate vector of size 135168 Kb

Does this mean that there is no way in which I can compute this distance
matrix on the machine I described below?

Is there any possible solution to this problem?

Thanks again to all,
Arnav.



----- Original Message -----
From: "Prof Brian Ripley" <ripley at stats.ox.ac.uk>
To: "Arnav Sheth" <sheth at economics.rutgers.edu>
Cc: "R-Help" <R-help at stat.math.ethz.ch>
Sent: Wednesday, February 25, 2004 2:12 PM
Subject: Re: [R] Computing very large distance matrix

On Wed, 25 Feb 2004, Arnav Sheth wrote:

Hello All,

I have a 131072x132 matrix for which I need to compute a regular

euclidean distance matrix, which I then need to transform and run agnes() on
this transformed matrix. I am having trouble computing the distance matrix
as it is fairly large and I am sure I have gone over the max.

The specific error I am getting is:
Error in vector("double", length) : negative length vectors are not

allowed

We've improved that message for the next release.

I have increased the memory limit to the maximum capacity of my hard
drive (which is around 20gb), with no success.

You'd need to increase the 32-bit limit of Windows, too ....

I am running the RGUI on Windows XP with 512 mb of RAM.

Would anyone have any suggestions as to how I can overcome this problem?
I would be most grateful for any help.

No. The distance matrix is 8Gb all by itself.

--
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595