bigglm "update" with ff
On Thu, 2 Apr 2009, Yue Sheng wrote:
Hi, since bigglm doesn't have update, I was wondering how to achieve something like (similar to the example in ff package manual using biglm):
bigglm() doesn't have update because it needs to iterate over the data and update() couldn't do that. Instead of calling update(), you have to supply a function that will load the next chunk of data, and that can be reset to start again at the beginning. There is an example on the help page, and the code for bigglm.data.frame gives another example, as do the SQLiteConnection and RODBC methods. -thomas
first <- TRUE
ffrowapply ({
if (first) {
first <- FALSE
fit <- bigglm(eqn, as.data.frame(bigdata[i1:i2,,drop=FALSE]), chunksize =
10000, family = binomial())
} else {
fit <- update(fit, as.data.frame(bigdata[i1:i2,,drop=FALSE]), chunksize =
10000, family = binomial())
}
}, X=bigdata, VERBOSE = TRUE, BATCHSIZE = nmax)
Many thanks.
Yuesheng
[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Thomas Lumley Assoc. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle