memory problem read.table v array (PR#9526)
There is nothing to reproduce here: we do not have 639.txt.
But note that read.table returns a data frame, and ?array has
data: a vector (including a list) giving data to fill the array.
so I do wonder if this is what you intended: you seem to have tried to
create an array list with 639*639 elements. It is certainly not the same
sort of object as array(0,dim=c(409600,1)).
Had 639.txt contained 639 rows of reals, your 'a' would be about
639*639*639*8 bytes, beyond the address space of your machine.
as.matrix(read.table("639.txt", header=FALSE)) might have been what you
are looking for.
On Thu, 22 Feb 2007, wlangdon at essex.ac.uk wrote:
Full_Name: bill langdon
Version: 2.4.1
OS: ubuntu
Submission from: (NULL) (155.245.58.159)
#WBL 22 Feb 2007 ubuntu
R.version
#platform i486-pc-linux-gnu
#arch i486
#os linux-gnu
#system i486, linux-gnu
#status
#major 2
#minor 4.1
#year 2006
#month 12
#day 18
#svn rev 40228
#language R
#version.string R version 2.4.1 (2006-12-18)
#if matrix "a" is created by array "vals" is created ok
#if matrix "a" is created by read.table,
#peak resource use (CPU, memory) by "array()" is excessive
#a = array(0,dim=c(409600,1)); #ok
#a = read.table("big.txt",header=FALSE); #all memory used
#a = read.table("639x639.txt",header=FALSE); #all memory used
#a = read.table("4096.txt",header=FALSE); #all memory used
#a = read.table("4096nocomment.txt",header=FALSE); #all memory used
#a = read.table("tiny.txt",header=FALSE); #ten lines ok
#a = read.table("1000.txt",header=FALSE); #all memory used
a = read.table("639.txt",header=FALSE); #uses 1.6047029GB
dim(a)
dd = 639;
vals = array(a,dim=c(dd,dd));
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595