glm-poisson fitting 400.000 records
D_Tomas <tomasmeca <at> hotmail.com> writes:
Hi, I am trying to fi a glm-poisson model to 400.000 records. I have tried biglm and glmulti but i have problems... can it really be the case that 400.000 are too many records??? I am thinking of using random samples of my dataset.....
"I have problems" isn't enough for us to diagnose. I tried this trivial example in base R:
d <- data.frame(x=runif(4e5),y=rpois(4e5,5)) system.time(glm(y~x,family=poisson,data=d,trace=TRUE))
Deviance = 438614.6 Iterations - 1 Deviance = 417968.2 Iterations - 2 Deviance = 417921.2 Iterations - 3 Deviance = 417921.2 Iterations - 4 user system elapsed 5.444 12.952 18.429 Can you give us a hint about what went wrong??