Skip to content
Prev 78 / 20628 Next
Original Raw

large dataset - lmer2 vector size specified is too large

florian bw 
Hi,

I want to fit mRNA expression data to sex.

I have the following values:
  expr:     expression value (for gene/person)
  affyID:   gene ID
  cephID: person ID
  sex

with 224 genes and 195 persons, therefore 43,680 data points. Both
with the nlme and the lme4 package i get errors. I tried it with R 2.4
and 2.5, and the newest package versions.

I have a 64-machine with 8GB RAM. Is the dataset simply too large? I
already cut it down and would actually be glad if I could do the
calculation with ~ 8000x250 data points.

Thank you for your help.

Florian Breitwieser
UNSW Sydney
Systems Biolgy

---------------------------------------------------
R version 2.5.0 Under development (unstable) (2007-02-26 r40806)
x86_64-unknown-linux-gnu

locale:
LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C

attached base packages:
[1] "stats"     "graphics"  "grDevices" "utils"     "datasets"  "methods"
[7] "base"

other attached packages:
       lme4      Matrix     lattice        nlme
"0.9975-13" "0.9975-11"   "0.14-16"    "3.1-79"


-----------------------------------------------------
Error in vector("double", length) : vector size specified is too large


------------------------------------------------------
Error: cannot allocate vector of size 4.7 Gb
used (Mb) gc trigger   (Mb)   max used   (Mb)
Ncells  829281 44.3    5714627  305.2   10890793  581.7
Vcells 1881820 14.4 1034019068 7889.0 1103035467 8415.5


Another time I got the following message:
*** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: lme.formula(expr ~ affyID * sex, random = ~affyID | cephID, data = ds.n)
 2: lme(expr ~ affyID * sex, random = ~affyID | cephID, data = ds.n)

Thread (6 messages)

florian bw large dataset - lmer2 vector size specified is too large Mar 1 Douglas Bates large dataset - lmer2 vector size specified is too large Mar 2 Gregor Gorjanc large dataset - lmer2 vector size specified is too large Mar 2 Douglas Bates large dataset - lmer2 vector size specified is too large Mar 2 Josh Wills large dataset - lmer2 vector size specified is too large Mar 2 Gregor Gorjanc large dataset - lmer2 vector size specified is too large Mar 2