using mclapply (multi core apply) to do matrix multiplication
On 07/02/12 12:02, Alaios wrote:
Thank you very much for your point... I hope I can find some easy to follow instructions as I do not have root permission
Me neither on pur cluster - but that won't stop you from compiling and installing R in your home directory. By doing this, you have even more control. Cheers and good luck, Rainer for the many cores system and our system administrator want
to have easy instructions to follow. Thanks a gain. ------------------------------------------------------------------------ *From:* Rainer M Krug <r.m.krug at gmail.com> *To:* Alaios <alaios at yahoo.com> *Cc:* Ernest Adrogu? <nfdisco at gmail.com>; "r-help at r-project.org" <r-help at r-project.org> *Sent:* Tuesday, February 7, 2012 11:44 AM *Subject:* Re: [R] using mclapply (multi core apply) to do matrix multiplication On 07/02/12 11:31, Alaios wrote:
> I would like to thank you Ernest for your answer. I guess that this > is gonna be faster as right now R only sees one core. In my work > there is a system with 64 cores and you can see only one working. If > I understand it right a [m,n][n,k] matrix multiplication can be split > into rows (from first matrice) and columns (from the second matrice) > and then combine all the local results of each cpu together.
You definitaly can go this way, but I would STRONGLY recommend to search for "parallel BLAS", check in the R-admin manual the section "Linear Algebra" which deals with BLAS et al, and e.g. http://www.r-bloggers.com/compiling-64-bit-r-2-10-1-with-mkl-in-linux/ My guess is that a paralelization on the C level in the BLAS et al. library will be MUCH faster then a paralelization on R level. Also, there is a R-sig-hpc mailing list for these kind of questions. Cheers, Rainer
> > Would that be too weird for mclapply to handle? > > B.R Alex > > > > ________________________________ From: Ernest > Adrogu?<nfdisco at gmail.com <mailto:nfdisco at gmail.com>> To:
r-help at r-project.org <mailto:r-help at r-project.org> Sent: Tuesday,
> February 7, 2012 11:02 AM Subject: Re: [R] using mclapply (multi core > apply) to do matrix multiplication > > 7-02-2012, 00:29 (-0800); Alaios escriu:
>> Dear all, I am trying to multiply three different matrices and >> each matrice is of size 16384,16384 the normal %*% multiplciation >> operator has not finished one day now. As I am running a system >> with many cores (and it seems that R is using only one of those) I >> would like to write fast a brief function that converts the typical >> for loops of a matrix multiplication to a set of lapply sets >> (mclapply uses the lapply syntax but it applies the work to many >> cores). >> >> If my thinking is correct , in the sense that this will speed up >> things a lot, I want you to help me covert the first matrix in >> rows the second in columns and convert those in a format that >> lapply would like to work with.
> > If I understand correctly, R uses a specialized library called BLAS > to do matrix multiplications. I doubt re-implementing the matrix > multiplication code at R-level would be any faster. What you can try > is replace BLAS with a multicore version of BLAS although it's not > easy if you have to compile it yourself. > > Also, you may try to re-think the problem you're trying to solve. > Maybe there's a different approach that is less > computation-intensive. > > > > > ______________________________________________ R-help at r-project.org
<mailto:R-help at r-project.org>
> mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do > read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
-- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer at krugs.de <mailto:Rainer at krugs.de> Skype: RMkrug
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer at krugs.de Skype: RMkrug