Debian/Ubuntu + threaded BLAS/ATLAS (solved)
On Mar 14, 2012, at 11:52 PM, Dirk Eddelbuettel wrote:
On 14 March 2012 at 23:09, Simon Urbanek wrote: | I thought I'll post this here since I could not find an answer anywhere I searched: how to setup R with threaded BLAS on Debian/Ubuntu. | | First, neither Debian nor Ubuntu come with optimized ATLAS binaries, simply because by definition they need to be optimized for a particular machine. The good news: it is easy to build: | | apt-get source atlas You could also go to newer Atlas sources at this point. | cd atlas-3.* | # less debian/README.Debian # read the docs - it's short | # sudo apt-get build-dep atlas # <- you may need this if you don't have all tools installed | fakeroot debian/rules custom "Yes but." Someone should write an atlas-helper package that deals with this and ... | That will give you (after a long while) a bunch of libatlas*.deb files that you can easily install with sudo dpkg -i | | Now if you simply configure R --with-blas --with-lapack it will work, and you'll get a nice and fast BLAS, but unfortunately it will be single-threaded: | | > m=matrix(rnorm(4e6),2e3) | > system.time(tcrossprod(m)) | user system elapsed | 0.684 0.012 0.697 | | The problem is that all those nice libraries that are handled by update-alternatives are just the single-threaded part of ATLAS. If you want to use the parallelized part, you'll need to add the libpt* libraries by hand. This worked for me: ... also this. Because if you do not deal well with update-alternatives, you loose the plug-and-play of installing and installing BLAS providing packages.
Believe me, I searched the net like a madman because I really didn't want to give this up. Half the reason I posted this here is to not forget it when I update R again ;). I really don't want to compile things manually on Debian - it defeats the point and makes upgrades a hell. I think the prober solution would be to simply include the shared libpt* version as well under the alternatives control - more precisely package libptatlas like we used to long time ago, so you could use it instead of libatlas. You could then switch blas to it. It is actually trivial to do, that's why I was searching so hard because I thought someone must have done it (also I found one thread that was hinting at this being present in some most recent packaging but I did not find it ...). I can easily hack up the building part to build libptatlas.so, but I don't know enough about how the alternatives system registers things...
For the gcbd package/vignette/paper I hacked together a plugin-package, locally built and hence optmised, but following the Atlas packaging: edd at max:~$ dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb Package: libatlas39 Source: atlas39 Version: 3.9.25-1 Architecture: amd64 Maintainer: Dirk Eddelbuettel <edd at debian.org> Installed-Size: 22656 Depends: libc6 (>= 2.4), libgcc1 (>= 1:4.1.1), libgfortran3 (>= 4.3) Suggests: libblas3gf, liblapack3gf Provides: libatlas.so.3gf, libblas.so.3gf, liblapack.so.3gf Section: libs Priority: optional Homepage: http://math-atlas.sourceforge.net/ Description: Automatically Tuned Linear Algebra Software, version 3.9.25 ATLAS is an approach for the automatic generation and optimization of numerical software. Currently ATLAS supplies optimized versions for the complete set of linear algebra kernels known as the Basic Linear Algebra Subroutines (BLAS), and a subset of the linear algebra routines in the LAPACK library. . The libraries in this package are built without any processor extension instructions, and should run on all processors of this general architecture, albeit less than optimally. . This package is a local package meant for run-time, built in a somewhat non-standard way off the Atlas 3.9.25 sources. edd at max:~$ Someone with interest and a little time should get together with Sylvestre and see ifwe could turn this into a helper package, or an experimental package, or ... | | --with-blas='/usr/lib/atlas-base/libptcblas.a /usr/lib/atlas-base/libptf77blas.a /usr/lib/atlas-base/libatlas.a -lpthread -lm' --with-lapack | | I opted for using the static version of ATLAS, but you can equally well use the shared one. | | > m=matrix(rnorm(4e6),2e3) | > system.time(tcrossprod(m)) | user system elapsed | 1.548 0.068 0.133 | | Note that by default it will use all cores. You can add "-t N" to the atlas configure flags in debian/rules if you want to change that (e.g. if you want to combine it with explicit parallelization). | | I'm simply sharing this in the hope that it will be helpful - please no flames ;). Very useful. It all helps -- Claudia's post was very good, your's here is too and I am just trying to move the cart further along. Dirk | Cheers, | Simon | | _______________________________________________ | R-sig-hpc mailing list | R-sig-hpc at r-project.org | https://stat.ethz.ch/mailman/listinfo/r-sig-hpc -- "Outside of a dog, a book is a man's best friend. Inside of a dog, it is too dark to read." -- Groucho Marx