Bioc developers! I've been exploring the Bioconductor nightly builds http://bioconductor.org/checkResults/ a bit using this in-development package. https://github.com/mtmorgan/BiocBuildReports This rpt <- report() filter_recent(rpt) %>% print(n = n()) summarizes the packages changed in the 24 hours before the current build snapshot, and their fate on the nightly builds > filter_recent(rpt) %>% print(n=nrow(.)) # A tibble: 18 x 5 package buildsrc checksrc push txt <chr> <fct> <fct> <fct> <chr> 1 AllelicImbalance ERROR skipped skipped skipped 2 ChIPpeakAnno ERROR skipped skipped skipped 3 GenomicScores ERROR skipped skipped skipped 4 genphen ERROR skipped skipped skipped 5 metavizr ERROR skipped skipped skipped 6 QUBIC ERROR skipped skipped skipped 7 Repitools OK ERROR skipped skipped 8 rtracklayer OK WARNINGS UNNEEDED same version exists in interna? 9 genoset OK WARNINGS YES new version is higher than in ? 10 IRanges OK WARNINGS YES new version is higher than in ? 11 LOLA OK WARNINGS YES new version is higher than in ? 12 DAPAR OK OK UNNEEDED same version exists in interna? 13 AnnotationHubData OK OK YES new version is higher than in ? 14 BEARscc OK OK YES new version is higher than in ? 15 ChemmineR OK OK YES new version is higher than in ? 16 DESeq2 OK OK YES new version is higher than in ? 17 Onassis OK OK YES new version is higher than in ? 18 synergyfinder OK OK YES new version is higher than in ? A couple of things strike me about this. - All the packages with 'ERROR' in the buildsrc or checksrc column failed to propagate because the commit, or a previous version, does not build or check. It is not a very productive development model to push broken commits to git.bioconductor.org, especially if the developer doesn't realize that the commits are broken. The best practice is to build and check packages locally, and then commit. R CMD build <YourPackage> R CMD check <YourPackage_<version>.tar.gz An even more robust approach is to clone the repository locally, so that the build and check are against only the committed changes and not miscellaneous files or versions hanging out in your source directory cd /tmp git clone /path/to/<YourPackage git> R CMD build <YourPackage> ... I would not suggest using devtools during this final stage; the build system doesn't use devtools, as devtools has its own ideas about options etc with which to build packages. The very careful would also use the build and check options found on the build report of each package, e.g., from http://bioconductor.org/checkResults/3.7/bioc-LATEST/rtracklayer/malbec2-buildsrc.html one might R CMD build --keep-empty-dirs --no-resave-data rtracklayer A common problem is that packages are built and checked locally using the wrong version of R and / or Bioconductor -- commits to the devel branch should be against the devel version of Bioconductor and (for the current release cycle) the devel version of R. http://bioconductor.org/developers/how-to/useDevel/ - two packages (rtracklayer, DAPAR) built and checked successfully, but did not propagate to the public repository. This is because they did not bump their version number according to policy http://bioconductor.org/developers/how-to/version-numbering/ While there might be reasons for pushing commits to Biocoductor without version bumps, this usually seems to be an oversight on the developer's part. Basically, every public version of a package should have a different version -- there is no shortage of version numbers. Martin This email message may contain legally privileged and/or...{{dropped:2}}
[Bioc-devel] Nightly builds
4 messages · Martin Morgan, Michael Love, Leonardo Collado Torres
I've found this Makefile from Laurent et al makes it a lot easier to do build/check, getting closer to the ease of devtools while still building and checking from command line: https://github.com/ComputationalProteomicsUnit/maker You do: git clone git at github.com:ComputationalProteomicsUnit/maker.git ln -s maker/Makefile . Then configure maker/Makefile for your package and point it to your local R-devel
Hi, I try to follow all the best practices but sometimes I end up relying on the Bioc nightly build. Anyhow, I personally use r-travis** for quick checks to detect broken commits instead of running R CMD check/build on my laptop. This way I can submit a new commit that fixes a broken commit before the Bioc nightly build in about the same time as using my laptop. r-travis doesn't always work as Bioc might have newer versions of dependencies or other issues (cache, timing, installation of deps), but it helps most of the time. A scenario where I definitely rely on the Bioc nightly build is when I can't reproduce the error/warning on my machine (with the latest setup). For example, Windows issues***. Note that I do run a quick local test of whatever bug I'm solving, but I might miss how my changes affected another unit test. Best, Leonardo ** Example setup https://github.com/leekgroup/derfinderPlot/blob/master/.travis.yml *** You can also use r-hub for testing in other OS including Windows but maybe not with the R version that bioc-devel needs and/or without being able to set useDevel(). Example https://github.com/LieberInstitute/jaffelab/blob/master/.travis.yml#L17-L18 I've only used it for that non-bioc package. On Wed, Feb 7, 2018 at 9:51 AM, Martin Morgan
<martin.morgan at roswellpark.org> wrote:
Bioc developers! I've been exploring the Bioconductor nightly builds http://bioconductor.org/checkResults/ a bit using this in-development package. https://github.com/mtmorgan/BiocBuildReports This rpt <- report() filter_recent(rpt) %>% print(n = n()) summarizes the packages changed in the 24 hours before the current build snapshot, and their fate on the nightly builds
filter_recent(rpt) %>% print(n=nrow(.))
# A tibble: 18 x 5 package buildsrc checksrc push txt <chr> <fct> <fct> <fct> <chr> 1 AllelicImbalance ERROR skipped skipped skipped 2 ChIPpeakAnno ERROR skipped skipped skipped 3 GenomicScores ERROR skipped skipped skipped 4 genphen ERROR skipped skipped skipped 5 metavizr ERROR skipped skipped skipped 6 QUBIC ERROR skipped skipped skipped 7 Repitools OK ERROR skipped skipped 8 rtracklayer OK WARNINGS UNNEEDED same version exists in interna? 9 genoset OK WARNINGS YES new version is higher than in ? 10 IRanges OK WARNINGS YES new version is higher than in ? 11 LOLA OK WARNINGS YES new version is higher than in ? 12 DAPAR OK OK UNNEEDED same version exists in interna? 13 AnnotationHubData OK OK YES new version is higher than in ? 14 BEARscc OK OK YES new version is higher than in ? 15 ChemmineR OK OK YES new version is higher than in ? 16 DESeq2 OK OK YES new version is higher than in ? 17 Onassis OK OK YES new version is higher than in ? 18 synergyfinder OK OK YES new version is higher than in ? A couple of things strike me about this. - All the packages with 'ERROR' in the buildsrc or checksrc column failed to propagate because the commit, or a previous version, does not build or check. It is not a very productive development model to push broken commits to git.bioconductor.org, especially if the developer doesn't realize that the commits are broken. The best practice is to build and check packages locally, and then commit. R CMD build <YourPackage> R CMD check <YourPackage_<version>.tar.gz An even more robust approach is to clone the repository locally, so that the build and check are against only the committed changes and not miscellaneous files or versions hanging out in your source directory cd /tmp git clone /path/to/<YourPackage git> R CMD build <YourPackage> ... I would not suggest using devtools during this final stage; the build system doesn't use devtools, as devtools has its own ideas about options etc with which to build packages. The very careful would also use the build and check options found on the build report of each package, e.g., from http://bioconductor.org/checkResults/3.7/bioc-LATEST/rtracklayer/malbec2-buildsrc.html one might R CMD build --keep-empty-dirs --no-resave-data rtracklayer A common problem is that packages are built and checked locally using the wrong version of R and / or Bioconductor -- commits to the devel branch should be against the devel version of Bioconductor and (for the current release cycle) the devel version of R. http://bioconductor.org/developers/how-to/useDevel/ - two packages (rtracklayer, DAPAR) built and checked successfully, but did not propagate to the public repository. This is because they did not bump their version number according to policy http://bioconductor.org/developers/how-to/version-numbering/ While there might be reasons for pushing commits to Biocoductor without version bumps, this usually seems to be an oversight on the developer's part. Basically, every public version of a package should have a different version -- there is no shortage of version numbers. Martin This email message may contain legally privileged and/or...
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
On Wed, Feb 7, 2018 at 10:36 AM, Leonardo Collado Torres
<lcollado at jhu.edu> wrote:
Hi, I try to follow all the best practices but sometimes I end up relying on the Bioc nightly build. Anyhow, I personally use r-travis** for quick checks to detect broken commits instead of running R CMD check/build on my laptop. This way I can submit a new commit that fixes a broken commit before the Bioc nightly build in about the same time as using my laptop. r-travis doesn't always work as Bioc might have newer versions of dependencies or other issues (cache, timing, installation of deps), but it helps most of the time. A scenario where I definitely rely on the Bioc nightly build is when I can't reproduce the error/warning on my machine (with the latest setup). For example, Windows issues***. Note that I do run a quick local test of whatever bug I'm solving, but I might miss how my changes affected another unit test. Best, Leonardo ** Example setup https://github.com/leekgroup/derfinderPlot/blob/master/.travis.yml *** You can also use r-hub for testing in other OS including Windows but maybe not with the R version that bioc-devel needs and/or without being able to set useDevel(). Example https://github.com/LieberInstitute/jaffelab/blob/master/.travis.yml#L17-L18 I've only used it for that non-bioc package.
More on r-hub and Bioc https://github.com/r-hub/rhub/issues/38 which I just found
On Wed, Feb 7, 2018 at 9:51 AM, Martin Morgan <martin.morgan at roswellpark.org> wrote:
Bioc developers! I've been exploring the Bioconductor nightly builds http://bioconductor.org/checkResults/ a bit using this in-development package. https://github.com/mtmorgan/BiocBuildReports This rpt <- report() filter_recent(rpt) %>% print(n = n()) summarizes the packages changed in the 24 hours before the current build snapshot, and their fate on the nightly builds
filter_recent(rpt) %>% print(n=nrow(.))
# A tibble: 18 x 5 package buildsrc checksrc push txt <chr> <fct> <fct> <fct> <chr> 1 AllelicImbalance ERROR skipped skipped skipped 2 ChIPpeakAnno ERROR skipped skipped skipped 3 GenomicScores ERROR skipped skipped skipped 4 genphen ERROR skipped skipped skipped 5 metavizr ERROR skipped skipped skipped 6 QUBIC ERROR skipped skipped skipped 7 Repitools OK ERROR skipped skipped 8 rtracklayer OK WARNINGS UNNEEDED same version exists in interna? 9 genoset OK WARNINGS YES new version is higher than in ? 10 IRanges OK WARNINGS YES new version is higher than in ? 11 LOLA OK WARNINGS YES new version is higher than in ? 12 DAPAR OK OK UNNEEDED same version exists in interna? 13 AnnotationHubData OK OK YES new version is higher than in ? 14 BEARscc OK OK YES new version is higher than in ? 15 ChemmineR OK OK YES new version is higher than in ? 16 DESeq2 OK OK YES new version is higher than in ? 17 Onassis OK OK YES new version is higher than in ? 18 synergyfinder OK OK YES new version is higher than in ? A couple of things strike me about this. - All the packages with 'ERROR' in the buildsrc or checksrc column failed to propagate because the commit, or a previous version, does not build or check. It is not a very productive development model to push broken commits to git.bioconductor.org, especially if the developer doesn't realize that the commits are broken. The best practice is to build and check packages locally, and then commit. R CMD build <YourPackage> R CMD check <YourPackage_<version>.tar.gz An even more robust approach is to clone the repository locally, so that the build and check are against only the committed changes and not miscellaneous files or versions hanging out in your source directory cd /tmp git clone /path/to/<YourPackage git> R CMD build <YourPackage> ... I would not suggest using devtools during this final stage; the build system doesn't use devtools, as devtools has its own ideas about options etc with which to build packages. The very careful would also use the build and check options found on the build report of each package, e.g., from http://bioconductor.org/checkResults/3.7/bioc-LATEST/rtracklayer/malbec2-buildsrc.html one might R CMD build --keep-empty-dirs --no-resave-data rtracklayer A common problem is that packages are built and checked locally using the wrong version of R and / or Bioconductor -- commits to the devel branch should be against the devel version of Bioconductor and (for the current release cycle) the devel version of R. http://bioconductor.org/developers/how-to/useDevel/ - two packages (rtracklayer, DAPAR) built and checked successfully, but did not propagate to the public repository. This is because they did not bump their version number according to policy http://bioconductor.org/developers/how-to/version-numbering/ While there might be reasons for pushing commits to Biocoductor without version bumps, this usually seems to be an oversight on the developer's part. Basically, every public version of a package should have a different version -- there is no shortage of version numbers. Martin This email message may contain legally privileged and/or...
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel