[Bioc-devel] Nightly builds

Hi,

I try to follow all the best practices but sometimes I end up relying
on the Bioc nightly build.

Anyhow, I personally use r-travis** for quick checks to detect broken
commits instead of running R CMD check/build on my laptop. This way I
can submit a new commit that fixes a broken commit before the Bioc
nightly build in about the same time as using my laptop. r-travis
doesn't always work as Bioc might have newer versions of dependencies
or other issues (cache, timing, installation of deps), but it helps
most of the time. A scenario where I definitely rely on the Bioc
nightly build is when I can't reproduce the error/warning on my
machine (with the latest setup). For example, Windows issues***.

Note that I do run a quick local test of whatever bug I'm solving, but
I might miss how my changes affected another unit test.

Best,
Leonardo

** Example setup
https://github.com/leekgroup/derfinderPlot/blob/master/.travis.yml
*** You can also use r-hub for testing in other OS including Windows
but maybe not with the R version that bioc-devel needs and/or without
being able to set useDevel(). Example
https://github.com/LieberInstitute/jaffelab/blob/master/.travis.yml#L17-L18
I've only used it for that non-bioc package.

On Wed, Feb 7, 2018 at 9:51 AM, Martin Morgan
<martin.morgan at roswellpark.org> wrote:

Bioc developers!

I've been exploring the Bioconductor nightly builds

  http://bioconductor.org/checkResults/

a bit using this in-development package.

  https://github.com/mtmorgan/BiocBuildReports

This

  rpt <- report()
  filter_recent(rpt) %>% print(n = n())

summarizes the packages changed in the 24 hours before the current build snapshot, and their fate on the nightly builds

filter_recent(rpt) %>% print(n=nrow(.))

# A tibble: 18 x 5
   package           buildsrc checksrc push     txt
   <chr>             <fct>    <fct>    <fct>    <chr>
 1 AllelicImbalance  ERROR    skipped  skipped  skipped
 2 ChIPpeakAnno      ERROR    skipped  skipped  skipped
 3 GenomicScores     ERROR    skipped  skipped  skipped
 4 genphen           ERROR    skipped  skipped  skipped
 5 metavizr          ERROR    skipped  skipped  skipped
 6 QUBIC             ERROR    skipped  skipped  skipped
 7 Repitools         OK       ERROR    skipped  skipped
 8 rtracklayer       OK       WARNINGS UNNEEDED same version exists in interna?
 9 genoset           OK       WARNINGS YES      new version is higher than in ?
10 IRanges           OK       WARNINGS YES      new version is higher than in ?
11 LOLA              OK       WARNINGS YES      new version is higher than in ?
12 DAPAR             OK       OK       UNNEEDED same version exists in interna?
13 AnnotationHubData OK       OK       YES      new version is higher than in ?
14 BEARscc           OK       OK       YES      new version is higher than in ?
15 ChemmineR         OK       OK       YES      new version is higher than in ?
16 DESeq2            OK       OK       YES      new version is higher than in ?
17 Onassis           OK       OK       YES      new version is higher than in ?
18 synergyfinder     OK       OK       YES      new version is higher than in ?

A couple of things strike me about this.

  - All the packages with 'ERROR' in the buildsrc or checksrc column failed to propagate because the commit, or a previous version, does not build or check. It is not a very productive development model to push broken commits to git.bioconductor.org, especially if the developer doesn't realize that the commits are broken. The best practice is to build and check packages locally, and then commit.

  R CMD build <YourPackage>
  R CMD check <YourPackage_<version>.tar.gz

An even more robust approach is to clone the repository locally, so that the build and check are against only the committed changes and not miscellaneous files or versions hanging out in your source directory

  cd /tmp
  git clone /path/to/<YourPackage git>
  R CMD build <YourPackage>
  ...

I would not suggest using devtools during this final stage; the build system doesn't use devtools, as devtools has its own ideas about options etc with which to build packages. The very careful would also use the build and check options found on the build report of each package, e.g., from


http://bioconductor.org/checkResults/3.7/bioc-LATEST/rtracklayer/malbec2-buildsrc.html

one might

  R CMD build --keep-empty-dirs --no-resave-data rtracklayer

A common problem is that packages are built and checked locally using the wrong version of R and / or Bioconductor -- commits to the devel branch should be against the devel version of Bioconductor and (for the current release cycle) the devel version of R.

  http://bioconductor.org/developers/how-to/useDevel/

  - two packages (rtracklayer, DAPAR) built and checked successfully, but did not propagate to the public repository. This is because they did not bump their version number according to policy

  http://bioconductor.org/developers/how-to/version-numbering/

While there might be reasons for pushing commits to Biocoductor without version bumps, this usually seems to be an oversight on the developer's part. Basically, every public version of a package should have a different version -- there is no shortage of version numbers.

Martin


This email message may contain legally privileged and/or...