q about memory usage progression
One more note. In case it is helpful, I am including the code for my loop:
# data is read in
numSNPs <- ncol(myData);
pvalues <- rep(-1,numSNPs);
names(pvalues) <- colnames(myData);
for (SNPnum in 1:numSNPs)
{
is.na(pvalues[SNPnum]) <- TRUE;
try({
fit.yags <- yags(log(myPhenos$PHENOTYPE) ~
myPhenos$AGE+myPhenos$SEX*myData[,SNPnum], id=myPhenos$id,
family=gaussian,corstr="exchangeable",alphainit=0.05)
z.gee <- fit.yags at coefficients[5]/sqrt(fit.yags at robust.parmvar[5,5]);
pval <- 2 * pnorm(abs(z.gee), lower.tail = FALSE);
pvalues[SNPnum] <- pval;
})
}
pvalues <- format(pvalues,digits=3);
On Mon, Dec 29, 2008 at 11:59 AM, Juliet Hannah <juliet.hannah at gmail.com> wrote:
I monitored the usage of memory on a script that I ran. It ran 30K regressions and it stores p-values for one of the coefficients. It read in a file that has 3000 rows and about 30K columns. The size of the file is about 170 MB. My understanding is that memory usage started out at 2.2G and went up to 23G: cpu=00:03:08, mem=172.75822 GBs, io=0.00000, vmem=2.224G, maxvmem=2.224G cpu=00:42:35, mem=29517.64894 GBs, io=0.00000, vmem=23.612G, maxvmem=23.612G I know very little about how memory works, but I thought the hardest part would be reading the file in. Could someone explain why there is such a substantial increase over the course of the script. Thanks, Juliet