Skip to content

[Bioc-devel] GenomicScores: irreproducible build error caused by gwascat

15 messages · Vincent Carey, Robert Castelo, Martin Morgan +1 more

Original
Robert Castelo
# 
hi,

my package GenomicScores is not building, see:

http://bioconductor.org/checkResults/devel/bioc-LATEST/GenomicScores/malbec1-buildsrc.html

apparently, it is breaking in the following lines of its vignette:

library(gwascat)
data(ebicat37)

which in the report from the bioc build machine says:

gwascat loaded.  Use makeCurrentGwascat() to extract current image.
  from EBI.  The data folder of this package has some legacy extracts.
Quitting from lines 404-408 (GenomicScores.Rmd)
Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
object 'ebicat37' not found
--- failed re-building ?GenomicScores.Rmd?

however, in my installation of current bioc-devel on R-4.0 with all 
packages up to date, GenomicScores builds fine and i cannot reproduce 
this error. below you can find my session information after the previous 
two instructions. the logs of 'gwascat' show changes in May 2nd that 
could be potentially responsible for this but the fact is that 'gwascat' 
is not building either and it does not seem that the changes propagate 
through the build system, its version is still 2.21.0, on which 
GenomicScores built without problems for the current release.

i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he 
has some more specific suggestion about this but any hint will be 
greatly appreciated.

thanks!!

sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS:?? /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so

locale:
 ?[1] LC_CTYPE=en_US.UTF8?????? LC_NUMERIC=C
 ?[3] LC_TIME=en_US.UTF8??????? LC_COLLATE=en_US.UTF8
 ?[5] LC_MONETARY=en_US.UTF8??? LC_MESSAGES=en_US.UTF8
 ?[7] LC_PAPER=en_US.UTF8?????? LC_NAME=C
 ?[9] LC_ADDRESS=C????????????? LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C

attached base packages:
[1] stats???? graphics? grDevices utils???? datasets methods?? base

other attached packages:
[1] gwascat_2.21.0 colorout_1.2-2

loaded via a namespace (and not attached):
 ?[1] Rcpp_1.0.4.6 lattice_0.20-41
 ?[3] prettyunits_1.1.1 Rsamtools_2.5.1
 ?[5] Biostrings_2.57.1 assertthat_0.2.1
 ?[7] digest_0.6.25 BiocFileCache_1.13.0
 ?[9] R6_2.4.1 GenomeInfoDb_1.25.1
[11] stats4_4.0.0 RSQLite_2.2.0
[13] httr_1.4.1 ggplot2_3.3.1
[15] pillar_1.4.4 zlibbioc_1.35.0
[17] rlang_0.4.6 GenomicFeatures_1.41.0
[19] progress_1.2.2 curl_4.3
[21] blob_1.2.1 S4Vectors_0.27.11
[23] Matrix_1.2-18 BiocParallel_1.23.0
[25] stringr_1.4.0 RCurl_1.98-1.2
[27] bit_1.1-15.2 biomaRt_2.45.0
[29] munsell_0.5.0 DelayedArray_0.15.1
[31] compiler_4.0.0 rtracklayer_1.49.3
[33] pkgconfig_2.0.3 askpass_1.1
[35] BiocGenerics_0.35.3 openssl_1.4.1
[37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
[39] tibble_3.0.1 GenomeInfoDbData_1.2.3
[41] IRanges_2.23.7 matrixStats_0.56.0
[43] XML_3.99-0.3 crayon_1.3.4
[45] dplyr_1.0.0 dbplyr_1.4.4
[47] GenomicAlignments_1.25.1 bitops_1.0-6
[49] rappdirs_0.3.1 grid_4.0.0
[51] gtable_0.3.0 lifecycle_0.2.0
[53] DBI_1.1.0 magrittr_1.5
[55] scales_1.1.1 stringi_1.4.6
[57] XVector_0.29.1 ellipsis_0.3.1
[59] generics_0.0.2 vctrs_0.3.0
[61] tools_4.0.0 bit64_0.9-7
[63] Biobase_2.49.0 glue_1.4.1
[65] purrr_0.3.4 hms_0.5.3
[67] parallel_4.0.0 colorspace_1.4-1
[69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
[71] memoise_1.1.0
Martin Morgan
# 
The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken

  https://bioconductor.org/checkResults/devel/bioc-LATEST/gwascat/

in a way that causes your package to fail

Martin

?On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces at r-project.org on behalf of robert.castelo at upf.edu> wrote:

    hi,

    my package GenomicScores is not building, see:

    http://bioconductor.org/checkResults/devel/bioc-LATEST/GenomicScores/malbec1-buildsrc.html

    apparently, it is breaking in the following lines of its vignette:

    library(gwascat)
    data(ebicat37)

    which in the report from the bioc build machine says:

    gwascat loaded.  Use makeCurrentGwascat() to extract current image.
      from EBI.  The data folder of this package has some legacy extracts.
    Quitting from lines 404-408 (GenomicScores.Rmd)
    Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
    object 'ebicat37' not found
    --- failed re-building ?GenomicScores.Rmd?

    however, in my installation of current bioc-devel on R-4.0 with all 
    packages up to date, GenomicScores builds fine and i cannot reproduce 
    this error. below you can find my session information after the previous 
    two instructions. the logs of 'gwascat' show changes in May 2nd that 
    could be potentially responsible for this but the fact is that 'gwascat' 
    is not building either and it does not seem that the changes propagate 
    through the build system, its version is still 2.21.0, on which 
    GenomicScores built without problems for the current release.

    i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he 
    has some more specific suggestion about this but any hint will be 
    greatly appreciated.

    thanks!!

    sessionInfo()
    R version 4.0.0 (2020-04-24)
    Platform: x86_64-pc-linux-gnu (64-bit)
    Running under: CentOS Linux 7 (Core)

    Matrix products: default
    BLAS:   /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
    LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so

    locale:
      [1] LC_CTYPE=en_US.UTF8       LC_NUMERIC=C
      [3] LC_TIME=en_US.UTF8        LC_COLLATE=en_US.UTF8
      [5] LC_MONETARY=en_US.UTF8    LC_MESSAGES=en_US.UTF8
      [7] LC_PAPER=en_US.UTF8       LC_NAME=C
      [9] LC_ADDRESS=C              LC_TELEPHONE=C
    [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C

    attached base packages:
    [1] stats     graphics  grDevices utils     datasets methods   base

    other attached packages:
    [1] gwascat_2.21.0 colorout_1.2-2

    loaded via a namespace (and not attached):
      [1] Rcpp_1.0.4.6 lattice_0.20-41
      [3] prettyunits_1.1.1 Rsamtools_2.5.1
      [5] Biostrings_2.57.1 assertthat_0.2.1
      [7] digest_0.6.25 BiocFileCache_1.13.0
      [9] R6_2.4.1 GenomeInfoDb_1.25.1
    [11] stats4_4.0.0 RSQLite_2.2.0
    [13] httr_1.4.1 ggplot2_3.3.1
    [15] pillar_1.4.4 zlibbioc_1.35.0
    [17] rlang_0.4.6 GenomicFeatures_1.41.0
    [19] progress_1.2.2 curl_4.3
    [21] blob_1.2.1 S4Vectors_0.27.11
    [23] Matrix_1.2-18 BiocParallel_1.23.0
    [25] stringr_1.4.0 RCurl_1.98-1.2
    [27] bit_1.1-15.2 biomaRt_2.45.0
    [29] munsell_0.5.0 DelayedArray_0.15.1
    [31] compiler_4.0.0 rtracklayer_1.49.3
    [33] pkgconfig_2.0.3 askpass_1.1
    [35] BiocGenerics_0.35.3 openssl_1.4.1
    [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
    [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
    [41] IRanges_2.23.7 matrixStats_0.56.0
    [43] XML_3.99-0.3 crayon_1.3.4
    [45] dplyr_1.0.0 dbplyr_1.4.4
    [47] GenomicAlignments_1.25.1 bitops_1.0-6
    [49] rappdirs_0.3.1 grid_4.0.0
    [51] gtable_0.3.0 lifecycle_0.2.0
    [53] DBI_1.1.0 magrittr_1.5
    [55] scales_1.1.1 stringi_1.4.6
    [57] XVector_0.29.1 ellipsis_0.3.1
    [59] generics_0.0.2 vctrs_0.3.0
    [61] tools_4.0.0 bit64_0.9-7
    [63] Biobase_2.49.0 glue_1.4.1
    [65] purrr_0.3.4 hms_0.5.3
    [67] parallel_4.0.0 colorspace_1.4-1
    [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
    [71] memoise_1.1.0




    _______________________________________________
    Bioc-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel
Robert Castelo
# 
never thought about it this way, but how can the system install 
something that does not build?

how should *i* install something that does not build to reproduce the error?

sorry if these are very naive questions!!

robert.
On 05/06/2020 13:34, Martin Morgan wrote:
Vincent Carey
# 
I am really sorry about the situation with gwascat and will try to
straighten it out today.

On Fri, Jun 5, 2020 at 6:27 AM Robert Castelo <robert.castelo at upf.edu>
wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

      
R CMD INSTALL <foo> or install.packages("foo", repos = NULL) on a clone of the repository. A build might fail because the examples or vignette fail to build, whereas for installation the only requirement is that the package is syntactically correct.

Martin

        
?On 6/5/20, 7:58 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:

        
never thought about it this way, but how can the system install 
    something that does not build?

    how should *i* install something that does not build to reproduce the error?

    sorry if these are very naive questions!!

    robert.

        
On 05/06/2020 13:34, Martin Morgan wrote:

        
> The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken
    >
    >    https://bioconductor.org/checkResults/devel/bioc-LATEST/gwascat/
    >
    > in a way that causes your package to fail
    >
    > Martin
    >
    > ?On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces at r-project.org on behalf of robert.castelo at upf.edu> wrote:
    >
    >      hi,
    >
    >      my package GenomicScores is not building, see:
    >
    >      http://bioconductor.org/checkResults/devel/bioc-LATEST/GenomicScores/malbec1-buildsrc.html
    >
    >      apparently, it is breaking in the following lines of its vignette:
    >
    >      library(gwascat)
    >      data(ebicat37)
    >
    >      which in the report from the bioc build machine says:
    >
    >      gwascat loaded.  Use makeCurrentGwascat() to extract current image.
    >        from EBI.  The data folder of this package has some legacy extracts.
    >      Quitting from lines 404-408 (GenomicScores.Rmd)
    >      Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
    >      object 'ebicat37' not found
    >      --- failed re-building ?GenomicScores.Rmd?
    >
    >      however, in my installation of current bioc-devel on R-4.0 with all
    >      packages up to date, GenomicScores builds fine and i cannot reproduce
    >      this error. below you can find my session information after the previous
    >      two instructions. the logs of 'gwascat' show changes in May 2nd that
    >      could be potentially responsible for this but the fact is that 'gwascat'
    >      is not building either and it does not seem that the changes propagate
    >      through the build system, its version is still 2.21.0, on which
    >      GenomicScores built without problems for the current release.
    >
    >      i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he
    >      has some more specific suggestion about this but any hint will be
    >      greatly appreciated.
    >
    >      thanks!!
    >
    >      sessionInfo()
    >      R version 4.0.0 (2020-04-24)
    >      Platform: x86_64-pc-linux-gnu (64-bit)
    >      Running under: CentOS Linux 7 (Core)
    >
    >      Matrix products: default
    >      BLAS:   /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
    >      LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so
    >
    >      locale:
    >        [1] LC_CTYPE=en_US.UTF8       LC_NUMERIC=C
    >        [3] LC_TIME=en_US.UTF8        LC_COLLATE=en_US.UTF8
    >        [5] LC_MONETARY=en_US.UTF8    LC_MESSAGES=en_US.UTF8
    >        [7] LC_PAPER=en_US.UTF8       LC_NAME=C
    >        [9] LC_ADDRESS=C              LC_TELEPHONE=C
    >      [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C
    >
    >      attached base packages:
    >      [1] stats     graphics  grDevices utils     datasets methods   base
    >
    >      other attached packages:
    >      [1] gwascat_2.21.0 colorout_1.2-2
    >
    >      loaded via a namespace (and not attached):
    >        [1] Rcpp_1.0.4.6 lattice_0.20-41
    >        [3] prettyunits_1.1.1 Rsamtools_2.5.1
    >        [5] Biostrings_2.57.1 assertthat_0.2.1
    >        [7] digest_0.6.25 BiocFileCache_1.13.0
    >        [9] R6_2.4.1 GenomeInfoDb_1.25.1
    >      [11] stats4_4.0.0 RSQLite_2.2.0
    >      [13] httr_1.4.1 ggplot2_3.3.1
    >      [15] pillar_1.4.4 zlibbioc_1.35.0
    >      [17] rlang_0.4.6 GenomicFeatures_1.41.0
    >      [19] progress_1.2.2 curl_4.3
    >      [21] blob_1.2.1 S4Vectors_0.27.11
    >      [23] Matrix_1.2-18 BiocParallel_1.23.0
    >      [25] stringr_1.4.0 RCurl_1.98-1.2
    >      [27] bit_1.1-15.2 biomaRt_2.45.0
    >      [29] munsell_0.5.0 DelayedArray_0.15.1
    >      [31] compiler_4.0.0 rtracklayer_1.49.3
    >      [33] pkgconfig_2.0.3 askpass_1.1
    >      [35] BiocGenerics_0.35.3 openssl_1.4.1
    >      [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
    >      [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
    >      [41] IRanges_2.23.7 matrixStats_0.56.0
    >      [43] XML_3.99-0.3 crayon_1.3.4
    >      [45] dplyr_1.0.0 dbplyr_1.4.4
    >      [47] GenomicAlignments_1.25.1 bitops_1.0-6
    >      [49] rappdirs_0.3.1 grid_4.0.0
    >      [51] gtable_0.3.0 lifecycle_0.2.0
    >      [53] DBI_1.1.0 magrittr_1.5
    >      [55] scales_1.1.1 stringi_1.4.6
    >      [57] XVector_0.29.1 ellipsis_0.3.1
    >      [59] generics_0.0.2 vctrs_0.3.0
    >      [61] tools_4.0.0 bit64_0.9-7
    >      [63] Biobase_2.49.0 glue_1.4.1
    >      [65] purrr_0.3.4 hms_0.5.3
    >      [67] parallel_4.0.0 colorspace_1.4-1
    >      [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
    >      [71] memoise_1.1.0
    >
    >
    >
    >      	[[alternative HTML version deleted]]
    >
    >      _______________________________________________
    >      Bioc-devel at r-project.org mailing list
    >      https://stat.ethz.ch/mailman/listinfo/bioc-devel

  
  


  


      

    

    

      
      

        


  

        

      Robert Castelo
    

    

      
      #
    

  


  

      
Thanks Martin, i wasn't aware about that fact. i've cloned the gwascat 
repo and been able to directly install from the directory and reproduce 
the error. in this case, this was easy because it involved just one 
package but with multiple broken package dependencies, i'd have to 
manually clone each of them and install them to reproduce the error.

would it be a good idea in the devel build system to propagate the tar 
ball resulting of

R CMD build --no-build-vignettes

so that we get automatically updates for at least those whose problem is 
in the vignettes only?

robert.

        
On 05/06/2020 15:37, Martin Morgan wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Robert Castelo
    

    

      
      #
    

  


  

      
hi Vince,

no worries, i've been finally able to reproduce the error in my computer 
and i'm afraid i'm probably using a deprecated dataset from 'gwascat', 
concretely the one that was stored as

gwascat/data/ebicat37.rda

what would be the replacement for the following kind of operation?

data(ebicat37)
eyersids <- unique(subsetByTraits(ebicat37, tr="Eye color")$SNPS)
eyersids
 ?[1] "rs12913832" "rs7173419"? "rs3002288"? "rs12896399" "rs1408799"
 ?[6] "rs12520016" "rs288139"?? "rs1667394"? "rs4596632" "rs12203592"
[11] "rs16891982" "rs1393350"? "rs1847134"

thanks!

robert.

        
On 05/06/2020 14:06, Vincent Carey wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

      
no, the build system should only propagate packages that have passed build and check -- the goal is to make the life of the user easier and more reliable, us developers have to sweat the details!

Usually there is a single package that has changed and causes problems, and like in the present case a little detective work (once one knows what to look for) leads quickly to the suspect.

Martin

        
?On 6/5/20, 10:18 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:

        
Thanks Martin, i wasn't aware about that fact. i've cloned the gwascat 
    repo and been able to directly install from the directory and reproduce 
    the error. in this case, this was easy because it involved just one 
    package but with multiple broken package dependencies, i'd have to 
    manually clone each of them and install them to reproduce the error.

    would it be a good idea in the devel build system to propagate the tar 
    ball resulting of

    R CMD build --no-build-vignettes

    so that we get automatically updates for at least those whose problem is 
    in the vignettes only?

    robert.

        
On 05/06/2020 15:37, Martin Morgan wrote:

        
> R CMD INSTALL <foo> or install.packages("foo", repos = NULL) on a clone of the repository. A build might fail because the examples or vignette fail to build, whereas for installation the only requirement is that the package is syntactically correct.
    >
    > Martin
    >

        
> ?On 6/5/20, 7:58 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:

        
>
    >      never thought about it this way, but how can the system install
    >      something that does not build?
    >
    >      how should *i* install something that does not build to reproduce the error?
    >
    >      sorry if these are very naive questions!!
    >
    >      robert.
    >

        
>      On 05/06/2020 13:34, Martin Morgan wrote:

        
>      > The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken
    >      >
    >      >    https://bioconductor.org/checkResults/devel/bioc-LATEST/gwascat/
    >      >
    >      > in a way that causes your package to fail
    >      >
    >      > Martin
    >      >
    >      > ?On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces at r-project.org on behalf of robert.castelo at upf.edu> wrote:
    >      >
    >      >      hi,
    >      >
    >      >      my package GenomicScores is not building, see:
    >      >
    >      >      http://bioconductor.org/checkResults/devel/bioc-LATEST/GenomicScores/malbec1-buildsrc.html
    >      >
    >      >      apparently, it is breaking in the following lines of its vignette:
    >      >
    >      >      library(gwascat)
    >      >      data(ebicat37)
    >      >
    >      >      which in the report from the bioc build machine says:
    >      >
    >      >      gwascat loaded.  Use makeCurrentGwascat() to extract current image.
    >      >        from EBI.  The data folder of this package has some legacy extracts.
    >      >      Quitting from lines 404-408 (GenomicScores.Rmd)
    >      >      Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
    >      >      object 'ebicat37' not found
    >      >      --- failed re-building ?GenomicScores.Rmd?
    >      >
    >      >      however, in my installation of current bioc-devel on R-4.0 with all
    >      >      packages up to date, GenomicScores builds fine and i cannot reproduce
    >      >      this error. below you can find my session information after the previous
    >      >      two instructions. the logs of 'gwascat' show changes in May 2nd that
    >      >      could be potentially responsible for this but the fact is that 'gwascat'
    >      >      is not building either and it does not seem that the changes propagate
    >      >      through the build system, its version is still 2.21.0, on which
    >      >      GenomicScores built without problems for the current release.
    >      >
    >      >      i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he
    >      >      has some more specific suggestion about this but any hint will be
    >      >      greatly appreciated.
    >      >
    >      >      thanks!!
    >      >
    >      >      sessionInfo()
    >      >      R version 4.0.0 (2020-04-24)
    >      >      Platform: x86_64-pc-linux-gnu (64-bit)
    >      >      Running under: CentOS Linux 7 (Core)
    >      >
    >      >      Matrix products: default
    >      >      BLAS:   /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
    >      >      LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so
    >      >
    >      >      locale:
    >      >        [1] LC_CTYPE=en_US.UTF8       LC_NUMERIC=C
    >      >        [3] LC_TIME=en_US.UTF8        LC_COLLATE=en_US.UTF8
    >      >        [5] LC_MONETARY=en_US.UTF8    LC_MESSAGES=en_US.UTF8
    >      >        [7] LC_PAPER=en_US.UTF8       LC_NAME=C
    >      >        [9] LC_ADDRESS=C              LC_TELEPHONE=C
    >      >      [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C
    >      >
    >      >      attached base packages:
    >      >      [1] stats     graphics  grDevices utils     datasets methods   base
    >      >
    >      >      other attached packages:
    >      >      [1] gwascat_2.21.0 colorout_1.2-2
    >      >
    >      >      loaded via a namespace (and not attached):
    >      >        [1] Rcpp_1.0.4.6 lattice_0.20-41
    >      >        [3] prettyunits_1.1.1 Rsamtools_2.5.1
    >      >        [5] Biostrings_2.57.1 assertthat_0.2.1
    >      >        [7] digest_0.6.25 BiocFileCache_1.13.0
    >      >        [9] R6_2.4.1 GenomeInfoDb_1.25.1
    >      >      [11] stats4_4.0.0 RSQLite_2.2.0
    >      >      [13] httr_1.4.1 ggplot2_3.3.1
    >      >      [15] pillar_1.4.4 zlibbioc_1.35.0
    >      >      [17] rlang_0.4.6 GenomicFeatures_1.41.0
    >      >      [19] progress_1.2.2 curl_4.3
    >      >      [21] blob_1.2.1 S4Vectors_0.27.11
    >      >      [23] Matrix_1.2-18 BiocParallel_1.23.0
    >      >      [25] stringr_1.4.0 RCurl_1.98-1.2
    >      >      [27] bit_1.1-15.2 biomaRt_2.45.0
    >      >      [29] munsell_0.5.0 DelayedArray_0.15.1
    >      >      [31] compiler_4.0.0 rtracklayer_1.49.3
    >      >      [33] pkgconfig_2.0.3 askpass_1.1
    >      >      [35] BiocGenerics_0.35.3 openssl_1.4.1
    >      >      [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
    >      >      [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
    >      >      [41] IRanges_2.23.7 matrixStats_0.56.0
    >      >      [43] XML_3.99-0.3 crayon_1.3.4
    >      >      [45] dplyr_1.0.0 dbplyr_1.4.4
    >      >      [47] GenomicAlignments_1.25.1 bitops_1.0-6
    >      >      [49] rappdirs_0.3.1 grid_4.0.0
    >      >      [51] gtable_0.3.0 lifecycle_0.2.0
    >      >      [53] DBI_1.1.0 magrittr_1.5
    >      >      [55] scales_1.1.1 stringi_1.4.6
    >      >      [57] XVector_0.29.1 ellipsis_0.3.1
    >      >      [59] generics_0.0.2 vctrs_0.3.0
    >      >      [61] tools_4.0.0 bit64_0.9-7
    >      >      [63] Biobase_2.49.0 glue_1.4.1
    >      >      [65] purrr_0.3.4 hms_0.5.3
    >      >      [67] parallel_4.0.0 colorspace_1.4-1
    >      >      [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
    >      >      [71] memoise_1.1.0
    >      >
    >      >
    >      >
    >      >      	[[alternative HTML version deleted]]
    >      >
    >      >      _______________________________________________
    >      >      Bioc-devel at r-project.org mailing list
    >      >      https://stat.ethz.ch/mailman/listinfo/bioc-devel
    >
    >

  
  


  


      

    

    

      
      

        


  

        

      Vincent Carey
    

    

      
      #
    

  


  

      
On Fri, Jun 5, 2020 at 10:24 AM Robert Castelo <robert.castelo at upf.edu>
wrote:

            

      
      
      
    

        
The best thing to do from a scientific point of view would likely be

curcat = gwascat::makeCurrentGwascat()

then work with that, which will have GRCh38 coordinates.

It is slow to run makeCurrentGwascat.  You might want to serialize what you
need.
I can't place a snapshot in gwascat as I had in the past because the image
compressed
with xz exceeds 5MB.  Any snapshot has to be in AnnotationHub.  I don't
have a
plan for this at the moment.

There is an object called ebicat_2020_04_30 in the data
folder of gwascat on devel branch.  It is a sample of 50000 records from
the full
catalog of that date.  You might be able to use that for what you are
doing; I will
be renaming it to clarify that it is a sample.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Robert Castelo
    

    

      
      #
    

  


  

      
Vince, thanks for the suggestions, i've taken the serializing route and 
submitted the fix.

cheers,

robert.

        
On 05/06/2020 17:28, Vincent Carey wrote:

            

      
      
      
    

  
  


  


      

    

    
3 days later

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 6/5/20 07:52, Martin Morgan wrote:

            

      
      
      
    

        
This makes a lot of sense for release but for devel maybe the criteria 
for propagating a package could be relaxed? It doesn't sound too insane 
to do that but I realize it does introduce a difference between release 
and devel that could also cause its own confusion.

Another approach that is currently under consideration is to make 
BiocManager::install() capable of installing/updating packages directly 
from git.bioconductor.org in BioC devel. This could be achieved by 
making the source tarballs resulting from

   git checkout master https://git.bioconductor.org/packages/<pkg> &&
     R CMD build --no-build-vignettes <pkg>

available in their own repo (e.g. 
https://bioconductor.org/packages/devel) and add this repo to 
BiocManager::repositories() in devel. Note that these no-vignettes 
source tarballs are actually those used by the build system during the 
INSTALL stage.

How do people feel about that?

H.

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 6/8/20 19:20, Herv? Pag?s wrote:

            

      
      
      
    

        
That URL (https://bioconductor.org/packages/devel) is already in use.

So maybe better at https://bioconductor.org/packages/master/ or 
something like that.

H.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Morgan
    

    

      
      #
    

  


  

      
I wrote a small gist

  https://gist.github.com/mtmorgan/21e18991c6ebdb388e8828bcc0fe72f6

that queries for broken dependencies. Using two packages that have been mentioned recently on this mailing list

  > tbl <- deps_broken(c("EnrichmentBrowser", "GenomicScores"))
  > tbl
  # A tibble: 1 x 5
    pkg               dep   install buildsrc checksrc
    <fct>             <chr> <chr>   <chr>    <chr>
  1 EnrichmentBrowser safe  OK      OK       ERROR

  > tbl %>% count(pkg, .drop=FALSE)
  # A tibble: 2 x 2
    pkg                   n
    <fct>             <int>
  1 EnrichmentBrowser     1
  2 GenomicScores         0

Trying to install 'safe' from public repositories fails

  > BiocManager::install("safe")
  Bioconductor version 3.12 (BiocManager 1.30.10), R 4.0.0 Patched (2020-05-14
    r78465)
  Installing package(s) 'safe'
  Warning message:
  package 'safe' is not available (for R version 4.0.0 Patched)

but it can be installed directly from git using the 'remotes' package

  remotes::install_git(
      "https://git.bioconductor.org/packages/safe",
      repos = BiocManager::repositories()
  )

I do not think it would be a good idea for BiocManager::install(version = "devel") to use a repository that included packages that installed but did not build or check. One interpretation of the GenomicScores exchange is that it helped alert the maintainer of gwascat about an error in their package, and encouraged a prompt fix.  Also BiocManager is really meant to help users follow best practices, rather than to make Bioconductor developer's lives easier. 

Perhaps there could be some other easy developer-oriented approach to installing broken packages?

Martin

        
?On 6/8/20, 10:20 PM, "Herv? Pag?s" <hpages at fredhutch.org> wrote:

        

        
On 6/5/20 07:52, Martin Morgan wrote:

        
> no, the build system should only propagate packages that have passed build and check -- the goal is to make the life of the user easier and more reliable, us developers have to sweat the details!

    This makes a lot of sense for release but for devel maybe the criteria 
    for propagating a package could be relaxed? It doesn't sound too insane 
    to do that but I realize it does introduce a difference between release 
    and devel that could also cause its own confusion.

    Another approach that is currently under consideration is to make 
    BiocManager::install() capable of installing/updating packages directly 
    from git.bioconductor.org in BioC devel. This could be achieved by 
    making the source tarballs resulting from

       git checkout master https://git.bioconductor.org/packages/<pkg> &&
         R CMD build --no-build-vignettes <pkg>

    available in their own repo (e.g. 
    https://bioconductor.org/packages/devel) and add this repo to 
    BiocManager::repositories() in devel. Note that these no-vignettes 
    source tarballs are actually those used by the build system during the 
    INSTALL stage.

    How do people feel about that?

    H.

    > 
    > Usually there is a single package that has changed and causes problems, and like in the present case a little detective work (once one knows what to look for) leads quickly to the suspect.
    > 
    > Martin
    >

        
> ?On 6/5/20, 10:18 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:

        
> 
    >      Thanks Martin, i wasn't aware about that fact. i've cloned the gwascat
    >      repo and been able to directly install from the directory and reproduce
    >      the error. in this case, this was easy because it involved just one
    >      package but with multiple broken package dependencies, i'd have to
    >      manually clone each of them and install them to reproduce the error.
    > 
    >      would it be a good idea in the devel build system to propagate the tar
    >      ball resulting of
    > 
    >      R CMD build --no-build-vignettes
    > 
    >      so that we get automatically updates for at least those whose problem is
    >      in the vignettes only?
    > 
    >      robert.
    >

        
>      On 05/06/2020 15:37, Martin Morgan wrote:

        
>      > R CMD INSTALL <foo> or install.packages("foo", repos = NULL) on a clone of the repository. A build might fail because the examples or vignette fail to build, whereas for installation the only requirement is that the package is syntactically correct.
    >      >
    >      > Martin
    >      >

        
>      > ?On 6/5/20, 7:58 AM, "Robert Castelo" <robert.castelo at upf.edu> wrote:

        
>      >
    >      >      never thought about it this way, but how can the system install
    >      >      something that does not build?
    >      >
    >      >      how should *i* install something that does not build to reproduce the error?
    >      >
    >      >      sorry if these are very naive questions!!
    >      >
    >      >      robert.
    >      >

        
>      >      On 05/06/2020 13:34, Martin Morgan wrote:

        
>      >      > The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken
    >      >      >
    >      >      >    https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_gwascat_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=r_GJJMDdjgc6nXTDSC0OhJW6gho041eSiWwKrw3h00w&e=
    >      >      >
    >      >      > in a way that causes your package to fail
    >      >      >
    >      >      > Martin
    >      >      >
    >      >      > ?On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces at r-project.org on behalf of robert.castelo at upf.edu> wrote:
    >      >      >
    >      >      >      hi,
    >      >      >
    >      >      >      my package GenomicScores is not building, see:
    >      >      >
    >      >      >      https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_GenomicScores_malbec1-2Dbuildsrc.html&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=-TxA290vxOPuMjHcxETokMk-jPL8H_YJpKMIr3UL50E&e=
    >      >      >
    >      >      >      apparently, it is breaking in the following lines of its vignette:
    >      >      >
    >      >      >      library(gwascat)
    >      >      >      data(ebicat37)
    >      >      >
    >      >      >      which in the report from the bioc build machine says:
    >      >      >
    >      >      >      gwascat loaded.  Use makeCurrentGwascat() to extract current image.
    >      >      >        from EBI.  The data folder of this package has some legacy extracts.
    >      >      >      Quitting from lines 404-408 (GenomicScores.Rmd)
    >      >      >      Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
    >      >      >      object 'ebicat37' not found
    >      >      >      --- failed re-building ?GenomicScores.Rmd?
    >      >      >
    >      >      >      however, in my installation of current bioc-devel on R-4.0 with all
    >      >      >      packages up to date, GenomicScores builds fine and i cannot reproduce
    >      >      >      this error. below you can find my session information after the previous
    >      >      >      two instructions. the logs of 'gwascat' show changes in May 2nd that
    >      >      >      could be potentially responsible for this but the fact is that 'gwascat'
    >      >      >      is not building either and it does not seem that the changes propagate
    >      >      >      through the build system, its version is still 2.21.0, on which
    >      >      >      GenomicScores built without problems for the current release.
    >      >      >
    >      >      >      i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he
    >      >      >      has some more specific suggestion about this but any hint will be
    >      >      >      greatly appreciated.
    >      >      >
    >      >      >      thanks!!
    >      >      >
    >      >      >      sessionInfo()
    >      >      >      R version 4.0.0 (2020-04-24)
    >      >      >      Platform: x86_64-pc-linux-gnu (64-bit)
    >      >      >      Running under: CentOS Linux 7 (Core)
    >      >      >
    >      >      >      Matrix products: default
    >      >      >      BLAS:   /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
    >      >      >      LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so
    >      >      >
    >      >      >      locale:
    >      >      >        [1] LC_CTYPE=en_US.UTF8       LC_NUMERIC=C
    >      >      >        [3] LC_TIME=en_US.UTF8        LC_COLLATE=en_US.UTF8
    >      >      >        [5] LC_MONETARY=en_US.UTF8    LC_MESSAGES=en_US.UTF8
    >      >      >        [7] LC_PAPER=en_US.UTF8       LC_NAME=C
    >      >      >        [9] LC_ADDRESS=C              LC_TELEPHONE=C
    >      >      >      [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C
    >      >      >
    >      >      >      attached base packages:
    >      >      >      [1] stats     graphics  grDevices utils     datasets methods   base
    >      >      >
    >      >      >      other attached packages:
    >      >      >      [1] gwascat_2.21.0 colorout_1.2-2
    >      >      >
    >      >      >      loaded via a namespace (and not attached):
    >      >      >        [1] Rcpp_1.0.4.6 lattice_0.20-41
    >      >      >        [3] prettyunits_1.1.1 Rsamtools_2.5.1
    >      >      >        [5] Biostrings_2.57.1 assertthat_0.2.1
    >      >      >        [7] digest_0.6.25 BiocFileCache_1.13.0
    >      >      >        [9] R6_2.4.1 GenomeInfoDb_1.25.1
    >      >      >      [11] stats4_4.0.0 RSQLite_2.2.0
    >      >      >      [13] httr_1.4.1 ggplot2_3.3.1
    >      >      >      [15] pillar_1.4.4 zlibbioc_1.35.0
    >      >      >      [17] rlang_0.4.6 GenomicFeatures_1.41.0
    >      >      >      [19] progress_1.2.2 curl_4.3
    >      >      >      [21] blob_1.2.1 S4Vectors_0.27.11
    >      >      >      [23] Matrix_1.2-18 BiocParallel_1.23.0
    >      >      >      [25] stringr_1.4.0 RCurl_1.98-1.2
    >      >      >      [27] bit_1.1-15.2 biomaRt_2.45.0
    >      >      >      [29] munsell_0.5.0 DelayedArray_0.15.1
    >      >      >      [31] compiler_4.0.0 rtracklayer_1.49.3
    >      >      >      [33] pkgconfig_2.0.3 askpass_1.1
    >      >      >      [35] BiocGenerics_0.35.3 openssl_1.4.1
    >      >      >      [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
    >      >      >      [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
    >      >      >      [41] IRanges_2.23.7 matrixStats_0.56.0
    >      >      >      [43] XML_3.99-0.3 crayon_1.3.4
    >      >      >      [45] dplyr_1.0.0 dbplyr_1.4.4
    >      >      >      [47] GenomicAlignments_1.25.1 bitops_1.0-6
    >      >      >      [49] rappdirs_0.3.1 grid_4.0.0
    >      >      >      [51] gtable_0.3.0 lifecycle_0.2.0
    >      >      >      [53] DBI_1.1.0 magrittr_1.5
    >      >      >      [55] scales_1.1.1 stringi_1.4.6
    >      >      >      [57] XVector_0.29.1 ellipsis_0.3.1
    >      >      >      [59] generics_0.0.2 vctrs_0.3.0
    >      >      >      [61] tools_4.0.0 bit64_0.9-7
    >      >      >      [63] Biobase_2.49.0 glue_1.4.1
    >      >      >      [65] purrr_0.3.4 hms_0.5.3
    >      >      >      [67] parallel_4.0.0 colorspace_1.4-1
    >      >      >      [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
    >      >      >      [71] memoise_1.1.0
    >      >      >
    >      >      >
    >      >      >
    >      >      >      	[[alternative HTML version deleted]]
    >      >      >
    >      >      >      _______________________________________________
    >      >      >      Bioc-devel at r-project.org mailing list
    >      >      >      https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
    >      >
    >      >
    > 
    > _______________________________________________
    > Bioc-devel at r-project.org mailing list
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
    > 

    -- 
    Herv? Pag?s

    Program in Computational Biology
    Division of Public Health Sciences
    Fred Hutchinson Cancer Research Center
    1100 Fairview Ave. N, M1-B514
    P.O. Box 19024
    Seattle, WA 98109-1024

    E-mail: hpages at fredhutch.org
    Phone:  (206) 667-5791
    Fax:    (206) 667-1319

  
  


  


      

    

    

      
      

        


  

        

      Robert Castelo
    

    

      
      #
    

  


  

      
Martin, i think this is a very good solution, with your script one 
identifies automatically the broken dependencies and then the 
installation via git avoid having to clone the corresponding repos. i 
tried it on another package with broken dependencies right now and it 
works very well:

deps_broken("GGtools")
# A tibble: 2 x 5
 ? pkg???? dep????????? install buildsrc checksrc
 ? <fct>?? <chr>??????? <chr> <chr>??? <chr>
1 GGtools GGBase?????? OK????? ERROR??? skipped
2 GGtools BiocParallel OK????? OK?????? TIMEOUT

i'd say the 'deps_broken()' function would be a very useful addition to 
BiocPkgTools ;)

robert.

        
On 09/06/2020 15:57, Martin Morgan wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 6/9/20 06:57, Martin Morgan wrote:

            

      
      
      
    

        
Cool! I never realized that remotes::install_git() could be used to 
install from arbitrary git servers but I guess that makes sense.

            

      
      
      
    

        
Well one could argue that the primary alert is the red glyphs on the 
build report.

            

      
      
      
    

        
Maybe via a different function then? E.g. BiocManager::install_master(). 
Would point to the master branch of software, data-exp and workflows, in 
addition to CRAN. Called with no argument it would be an easy way for 
developers to get their packages in sync with what's on the build 
machines so would make it easier to reproduce the errors they see on the 
build report.

H.