Parallel or 64-bit PC with a big RAM? [SEC=UNCLASSIFIED]
Hi Jin, The model fitting problem would seem small enough for most computers and methods. You could do the model prediction with the 'raster' package that was designed to be able to handle very large datasets. See ?raster::predict and raster::interpolate. Perhaps also consult the 'sdm' vignette for the 'dismo' package on R-Forge; it has examples (in a species distribution modeling context) for the same methods you mention. Best, Robert
On Mon, Jul 19, 2010 at 4:53 PM, <Jin.Li at ga.gov.au> wrote:
Dear All, A group of us here need to make spatial predictions of environmental variables using methods in gstat, SVM in e1071, randomForest in randomForest in R. The dataset for model development may contain up to 80 variables with up to 15,000 rows and the dataset for prediction may contain up to 80 variables with up to 1 billion rows. The figures given here are extremes we are expecting, and of course we can reduce the variables for some methods but for some we need to use all variables. It seems we have two solutions at the moment: 1) run the modelling work on a cluster of computers and 2) run 64-bit R on a 64-bit computer for Windows with a big RAM. The questions are: can we do parallel computing using the packages listed if we choose option one? Or how big RAM do we need if we go option two? Any suggestions and comments are appreciated. Thanks in advance. Cheers, Jin
____________________________________ Jin Li, PhD Spatial Modeller/Computational Statistician Marine & Coastal Environment Geoscience Australia GPO Box 378, Canberra, ACT 2601, Australia Ph: 61 (02) 6249 9899; email: jin.li at ga.gov.au<mailto:jin.li at ga.gov.au> _______________________________________ ? ? ? ?[[alternative HTML version deleted]] _______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo