[Bioc-devel] PPA with built bioconductor packages (for continuous integration)
----- Original Message -----
From: "Martin Morgan" <mtmorgan at fredhutch.org> To: "Laurent Gautier" <lgautier at gmail.com>, bioc-devel at r-project.org Sent: Sunday, November 9, 2014 8:26:48 AM Subject: Re: [Bioc-devel] PPA with built bioconductor packages (for continuous integration) On 11/09/2014 07:23 AM, Laurent Gautier wrote:
Hi, Continuous integration is a convenient way to automate some of the steps necessary to ensure quality software. Popular ways to do it create a vanilla virtual machine 9VM) with a Linux distribution, and scripts prepares the VM with 3rd-party dependencies required by the software. For example, the popular CI system Travis for github creates by default a VM running ubuntu, and dependencies can be installed with `apt-get install`. When developing software that requires CRAN/bioconductor, the latest R is available precompiled but the R packages must be downloaded installed from source. This can take a relatively long time. On a recent project over 80% of the time is spent downloading/installing the R/BioC packages. The remaining is building the code and running the unit tests. Having a Personal Package Archive (PPA) with bioconductor packages already compiled would both speed up the process and make the use of continuous integration by projects relying on bioconductor packages easier. Is this something others would like to have, and is this something that bioconductor would see to its mission to provide / help provide quality software and be able to host ?
It would be interesting to catalogue objectives (e.g., development vs. reproducibility) and available alternatives (e.g., PPA, docker / Rocker, AMI, existing or possible cloud services [such as the Bioc 'single package builder' used to build and check new package submissions, or travis itself], the Becker repository management scheme Michael and Gabe mention, ...);
Just to add to the mix of options, it's possible to run R CMD INSTALL --build on a source tarball on Linux and it will create a 'binary' version that is already compiled. The problem with this is (AFAIK) there is no corresponding package type that can be used with install.packages(); otherwise the simplest solution would be to add a CRAN-style repos containing these "binaries". Maybe R could be patched to allow this? But it's possible that the requirements for Linux "binaries" could vary depending on many things: cpu type (intel or solaris, or...), architecture (i386, x64), presence/absence of BLAS/LAPACK, etc etc etc. This suggests that a vm or container-based approach might be better. Dan
if there is a clear path forward satisfying some plurality of users without too many technical obstacles then it might fall within the Bioc purview; my initial sense is that there is not a consensus on use cases or viable implementations, but I can be convinced otherwise... In terms of Tim's post, getting your colleague to use a PPA / existing alternative (e.g., the Bioc AMI, http://bioconductor.org/help/bioconductor-cloud-ami/ which comes with Rstudio server installed...) is not likely to be easier / faster than getting them to download / install relevant R / Bioc packages. One interesting possibility is a 'hosted' bioconductor with sufficient computational resources on the back-end and Rstudio server on the front end; this is not impossible to imaging seeking funding for.
Martin
Best, Laurent [[alternative HTML version deleted]]
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
-- Computational Biology / Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: Arnold Building M1 B861 Phone: (206) 667-2793
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel