Skip to content

[Bioc-devel] Using BioC vignettes as Jupyter Notebooks

3 messages · Martin Morgan, Neumann, Steffen

Original
Neumann, Steffen
# 
Hi,

last week over a beer some of us realized that (technically)
it is possible to run most[1] Rmd vignettes as Jupiter Notebook. 

So I created https://github.com/sneumann/bioc_notebooks 
which takes some example packages and creates a Docker container 
with converted vignettes. The resulting container 
can be hosted locally, or on the binder system (see README).

We found this to be fun, and it could be used 
as a great training resource. Only downside is 
that the resulting containers are not exactly light-weight.

Yours,
Steffen

[1] See the issue in the repo where it fails.
Martin Morgan
# 
Do you think this approach could be re-cast in the same way that https://github.com/Bioconductor/bioconductor_full has done, so that the container has all the software to _support_ the packages needed for the jupyter notebook, but that the user must do some kind of initialization to install the R libraries? In this way there would be one container instead of 1750, and it would not be heavy?

One idea that we've been exploring is making _binary_ versions of Bioc packages available that are compatible with bioconductor_full, so that package installation would sometimes be much faster (a simple download, rather than compilation).

Maritn

?On 7/2/19, 10:52 AM, "Bioc-devel on behalf of Neumann, Steffen" <bioc-devel-bounces at r-project.org on behalf of sneumann at ipb-halle.de> wrote:

    Hi,
    
    last week over a beer some of us realized that (technically)
    it is possible to run most[1] Rmd vignettes as Jupiter Notebook. 
    
    So I created https://github.com/sneumann/bioc_notebooks 
    which takes some example packages and creates a Docker container 
    with converted vignettes. The resulting container 
    can be hosted locally, or on the binder system (see README).
    
    We found this to be fun, and it could be used 
    as a great training resource. Only downside is 
    that the resulting containers are not exactly light-weight.
    
    Yours,
    Steffen
    
    [1] See the issue in the repo where it fails. 
    
    -- 
    
    IPB Halle                    AG Massenspektrometrie & Bioinformatik
    Dr. Steffen Neumann          http://www.IPB-Halle.DE
    Weinberg 3                   Tel. +49 (0) 345 5582 - 1470
    06120 Halle                       +49 (0) 345 5582 - 0
    sneumann(at)IPB-Halle.DE     Fax. +49 (0) 345 5582 - 1409
    _______________________________________________
    Bioc-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel
Neumann, Steffen
# 
Hi,
On Tue, 2019-07-02 at 09:50 +0000, Martin Morgan wrote:
Hm, the Jupyter can not "talk" to the container that is hosting it, 
so this would have to be done upon startup.
I like that idea to some extend. In particular, 
it'll help not having to carry the BUILD dependencies 
in the container as we currently have in e.g. release_metabolomics2.

A less known feature of Docker is to have two FROM in one Dockerfile, 
e
.g. https://github.com/ProteoWizard/container/blob/master/Dockerfile
where the second FROM can COPY from the above-built container, 
thus dropping any cruft. 

But how would the binary packages then be imported into 
the container ? The volumes work fine on the command line. 
What about a kubernetes cluster ? 

I'd like to suggest to avoid any dependency on the 
filesystem of the host, i.e. no 
-v host-site-library:/usr/local/lib/R/host-site-library

An established way to mount stuff into a container (like bioc-full)
is to use sidecar containers. Basically they share some of their 
filesystem with the "main" container (typical example: 
tomcat main container, sidecar with only the *.war file).

A challenge is duplication. If each binary with required /site-lib/content 
carries Biobase, we'd have 1750 copies of that. This would be solved if we can mount 
the required subset (=dependencies) of the 1750 binaries into site/lib/PACKAGE

Yours,
Steffen