[Bioc-devel] GenomicScores: irreproducible build error caused by gwascat
On 6/9/20 06:57, Martin Morgan wrote:
I wrote a small gist https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_mtmorgan_21e18991c6ebdb388e8828bcc0fe72f6&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-O2S2vunnN1XaaLDV0ShKsYOmdWVHU3VPC5w9GUdeDI&s=HWmLUbVyv6NbkJJ97AdGYpmXeit0Ip7ZLCU34ZB0Pqo&e= that queries for broken dependencies. Using two packages that have been mentioned recently on this mailing list
> tbl <- deps_broken(c("EnrichmentBrowser", "GenomicScores"))
> tbl
# A tibble: 1 x 5
pkg dep install buildsrc checksrc
<fct> <chr> <chr> <chr> <chr>
1 EnrichmentBrowser safe OK OK ERROR
> tbl %>% count(pkg, .drop=FALSE)
# A tibble: 2 x 2
pkg n
<fct> <int>
1 EnrichmentBrowser 1
2 GenomicScores 0
Trying to install 'safe' from public repositories fails
> BiocManager::install("safe")
Bioconductor version 3.12 (BiocManager 1.30.10), R 4.0.0 Patched (2020-05-14
r78465)
Installing package(s) 'safe'
Warning message:
package 'safe' is not available (for R version 4.0.0 Patched)
but it can be installed directly from git using the 'remotes' package
Cool! I never realized that remotes::install_git() could be used to install from arbitrary git servers but I guess that makes sense.
remotes::install_git(
"https://urldefense.proofpoint.com/v2/url?u=https-3A__git.bioconductor.org_packages_safe&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-O2S2vunnN1XaaLDV0ShKsYOmdWVHU3VPC5w9GUdeDI&s=erDWnwZtrMpHgSrJJ4Z219hyyi9K5Xq2cJVgtRBP9qU&e= ",
repos = BiocManager::repositories()
)
I do not think it would be a good idea for BiocManager::install(version = "devel") to use a repository that included packages that installed but did not build or check. One interpretation of the GenomicScores exchange is that it helped alert the maintainer of gwascat about an error in their package, and encouraged a prompt fix.
Well one could argue that the primary alert is the red glyphs on the build report.
Also BiocManager is really meant to help users follow best practices, rather than to make Bioconductor developer's lives easier. Perhaps there could be some other easy developer-oriented approach to installing broken packages?
Maybe via a different function then? E.g. BiocManager::install_master(). Would point to the master branch of software, data-exp and workflows, in addition to CRAN. Called with no argument it would be an easy way for developers to get their packages in sync with what's on the build machines so would make it easier to reproduce the errors they see on the build report. H.
Martin
?On 6/8/20, 10:20 PM, "Herv? Pag?s" <hpages at fredhutch.org> wrote:
On 6/5/20 07:52, Martin Morgan wrote:
> no, the build system should only propagate packages that have passed build and check -- the goal is to make the life of the user easier and more reliable, us developers have to sweat the details!
This makes a lot of sense for release but for devel maybe the criteria
for propagating a package could be relaxed? It doesn't sound too insane
to do that but I realize it does introduce a difference between release
and devel that could also cause its own confusion.
Another approach that is currently under consideration is to make
BiocManager::install() capable of installing/updating packages directly
from git.bioconductor.org in BioC devel. This could be achieved by
making the source tarballs resulting from
git checkout master https://urldefense.proofpoint.com/v2/url?u=https-3A__git.bioconductor.org_packages_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-O2S2vunnN1XaaLDV0ShKsYOmdWVHU3VPC5w9GUdeDI&s=byFAKeaah2EIS-mdsi3iT0Bek2Y-SqLNX5M007ZT82w&e= <pkg> &&
R CMD build --no-build-vignettes <pkg>
available in their own repo (e.g.
https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_packages_devel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-O2S2vunnN1XaaLDV0ShKsYOmdWVHU3VPC5w9GUdeDI&s=oIWaJnqFZ1SL9SIDDDc5IChaAr0NtnraSpAcJSHcBw4&e= ) and add this repo to
BiocManager::repositories() in devel. Note that these no-vignettes
source tarballs are actually those used by the build system during the
INSTALL stage.
How do people feel about that?
H.
>
> Usually there is a single package that has changed and causes problems, and like in the present case a little detective work (once one knows what to look for) leads quickly to the suspect.
>
> Martin
>
> ?On 6/5/20, 10:18 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:
>
> Thanks Martin, i wasn't aware about that fact. i've cloned the gwascat
> repo and been able to directly install from the directory and reproduce
> the error. in this case, this was easy because it involved just one
> package but with multiple broken package dependencies, i'd have to
> manually clone each of them and install them to reproduce the error.
>
> would it be a good idea in the devel build system to propagate the tar
> ball resulting of
>
> R CMD build --no-build-vignettes
>
> so that we get automatically updates for at least those whose problem is
> in the vignettes only?
>
> robert.
>
> On 05/06/2020 15:37, Martin Morgan wrote:
> > R CMD INSTALL <foo> or install.packages("foo", repos = NULL) on a clone of the repository. A build might fail because the examples or vignette fail to build, whereas for installation the only requirement is that the package is syntactically correct.
> >
> > Martin
> >
> > ?On 6/5/20, 7:58 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:
> >
> > never thought about it this way, but how can the system install
> > something that does not build?
> >
> > how should *i* install something that does not build to reproduce the error?
> >
> > sorry if these are very naive questions!!
> >
> > robert.
> >
> > On 05/06/2020 13:34, Martin Morgan wrote:
> > > The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken
> > >
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_gwascat_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=r_GJJMDdjgc6nXTDSC0OhJW6gho041eSiWwKrw3h00w&e=
> > >
> > > in a way that causes your package to fail
> > >
> > > Martin
> > >
> > > ?On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces at r-project.org on behalf of robert.castelo at upf.edu> wrote:
> > >
> > > hi,
> > >
> > > my package GenomicScores is not building, see:
> > >
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_GenomicScores_malbec1-2Dbuildsrc.html&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=-TxA290vxOPuMjHcxETokMk-jPL8H_YJpKMIr3UL50E&e=
> > >
> > > apparently, it is breaking in the following lines of its vignette:
> > >
> > > library(gwascat)
> > > data(ebicat37)
> > >
> > > which in the report from the bioc build machine says:
> > >
> > > gwascat loaded. Use makeCurrentGwascat() to extract current image.
> > > from EBI. The data folder of this package has some legacy extracts.
> > > Quitting from lines 404-408 (GenomicScores.Rmd)
> > > Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
> > > object 'ebicat37' not found
> > > --- failed re-building ?GenomicScores.Rmd?
> > >
> > > however, in my installation of current bioc-devel on R-4.0 with all
> > > packages up to date, GenomicScores builds fine and i cannot reproduce
> > > this error. below you can find my session information after the previous
> > > two instructions. the logs of 'gwascat' show changes in May 2nd that
> > > could be potentially responsible for this but the fact is that 'gwascat'
> > > is not building either and it does not seem that the changes propagate
> > > through the build system, its version is still 2.21.0, on which
> > > GenomicScores built without problems for the current release.
> > >
> > > i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he
> > > has some more specific suggestion about this but any hint will be
> > > greatly appreciated.
> > >
> > > thanks!!
> > >
> > > sessionInfo()
> > > R version 4.0.0 (2020-04-24)
> > > Platform: x86_64-pc-linux-gnu (64-bit)
> > > Running under: CentOS Linux 7 (Core)
> > >
> > > Matrix products: default
> > > BLAS: /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
> > > LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so
> > >
> > > locale:
> > > [1] LC_CTYPE=en_US.UTF8 LC_NUMERIC=C
> > > [3] LC_TIME=en_US.UTF8 LC_COLLATE=en_US.UTF8
> > > [5] LC_MONETARY=en_US.UTF8 LC_MESSAGES=en_US.UTF8
> > > [7] LC_PAPER=en_US.UTF8 LC_NAME=C
> > > [9] LC_ADDRESS=C LC_TELEPHONE=C
> > > [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C
> > >
> > > attached base packages:
> > > [1] stats graphics grDevices utils datasets methods base
> > >
> > > other attached packages:
> > > [1] gwascat_2.21.0 colorout_1.2-2
> > >
> > > loaded via a namespace (and not attached):
> > > [1] Rcpp_1.0.4.6 lattice_0.20-41
> > > [3] prettyunits_1.1.1 Rsamtools_2.5.1
> > > [5] Biostrings_2.57.1 assertthat_0.2.1
> > > [7] digest_0.6.25 BiocFileCache_1.13.0
> > > [9] R6_2.4.1 GenomeInfoDb_1.25.1
> > > [11] stats4_4.0.0 RSQLite_2.2.0
> > > [13] httr_1.4.1 ggplot2_3.3.1
> > > [15] pillar_1.4.4 zlibbioc_1.35.0
> > > [17] rlang_0.4.6 GenomicFeatures_1.41.0
> > > [19] progress_1.2.2 curl_4.3
> > > [21] blob_1.2.1 S4Vectors_0.27.11
> > > [23] Matrix_1.2-18 BiocParallel_1.23.0
> > > [25] stringr_1.4.0 RCurl_1.98-1.2
> > > [27] bit_1.1-15.2 biomaRt_2.45.0
> > > [29] munsell_0.5.0 DelayedArray_0.15.1
> > > [31] compiler_4.0.0 rtracklayer_1.49.3
> > > [33] pkgconfig_2.0.3 askpass_1.1
> > > [35] BiocGenerics_0.35.3 openssl_1.4.1
> > > [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
> > > [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
> > > [41] IRanges_2.23.7 matrixStats_0.56.0
> > > [43] XML_3.99-0.3 crayon_1.3.4
> > > [45] dplyr_1.0.0 dbplyr_1.4.4
> > > [47] GenomicAlignments_1.25.1 bitops_1.0-6
> > > [49] rappdirs_0.3.1 grid_4.0.0
> > > [51] gtable_0.3.0 lifecycle_0.2.0
> > > [53] DBI_1.1.0 magrittr_1.5
> > > [55] scales_1.1.1 stringi_1.4.6
> > > [57] XVector_0.29.1 ellipsis_0.3.1
> > > [59] generics_0.0.2 vctrs_0.3.0
> > > [61] tools_4.0.0 bit64_0.9-7
> > > [63] Biobase_2.49.0 glue_1.4.1
> > > [65] purrr_0.3.4 hms_0.5.3
> > > [67] parallel_4.0.0 colorspace_1.4-1
> > > [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
> > > [71] memoise_1.1.0
> > >
> > >
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > _______________________________________________
> > > Bioc-devel at r-project.org mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
> >
> >
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
>
--
Herv? Pag?s
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpages at fredhutch.org
Phone: (206) 667-5791
Fax: (206) 667-1319
Herv? Pag?s Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpages at fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319