Skip to content

[Bioc-devel] SummarizedExperiment assays() gives HDF5Array error in Rstudio

5 messages · Shraddha Pai, Vincent Carey, Marcel Ramos

#
Hello BioC community,

Calling assays() on a SummarizedExperiment object has suddenly started
giving me this error in Rstudio but not on command-line R. I have been
trying to solve this for a few hours without luck and any help would be
great. sessionInfo() output below and this is Rstudio 1.4.

Thanks,Shraddha

Browse[1]> brcaA MultiAssayExperiment object of 4 listed
 experiments with user-defined names and respective classes.
 Containing an ExperimentList class object of length 4:
 [1] BRCA_miRNASeqGene-20160128: SummarizedExperiment with 1046 rows
and 166 columns
 [2] BRCA_mRNAArray-20160128: SummarizedExperiment with 17814 rows and
453 columns
 [3] BRCA_RPPAArray-20160128: SummarizedExperiment with 226 rows and 362 columns
 [4] BRCA_Methylation_methyl27-20160128: SummarizedExperiment with
27578 rows and 261 columns
Functionality:
 experiments() - obtain the ExperimentList instance
 colData() - the primary/phenotype DataFrame
 sampleMap() - the sample coordination DataFrame
 `$`, `[`, `[[` - extract colData columns, subset, or experiment
 *Format() - convert into a long or wide DataFrame
 assays() - convert ExperimentList to a SimpleList of matrices
 exportClass() - save all data to files


Browse[1]> assays(experiments(brca)[[4]])Error in
.Call2("C_h5getdimscales", filepath, name, scalename, PACKAGE =
"HDF5Array") :
  failed to open file '/root/.cache/ExperimentHub/661a6f10d826_2122'


Browse[2]> experiments(brca)[[4]]class: SummarizedExperiment
dim: 27578 261
metadata(0):
assays(1): counts
rownames(27578): cg00000292 cg00002426 ... cg27662877 cg27665659
rowData names(3): Gene_Symbol Chromosome Genomic_Coordinate
colnames(261): TCGA-A1-A0SD-01A-11D-A112-05 TCGA-A2-A04N-01A-11D-A112-05 ...
  TCGA-E2-A15S-01A-11D-A112-05 TCGA-E2-A15T-01A-11D-A112-05
colData names(0):
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
LC_MESSAGES=C
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
LC_ADDRESS=C
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils
datasets  methods
[9] base

other attached packages:
 [1] rhdf5_2.34.0                netDx_1.2.3
 [3] bigmemory_4.5.36            curatedTCGAData_1.12.1
 [5] MultiAssayExperiment_1.16.0 SummarizedExperiment_1.20.0
 [7] Biobase_2.50.0              GenomicRanges_1.42.0
 [9] GenomeInfoDb_1.26.7         IRanges_2.24.1
[11] S4Vectors_0.28.1            BiocGenerics_0.36.1
[13] MatrixGenerics_1.2.1        matrixStats_0.58.0

loaded via a namespace (and not attached):
  [1] uuid_0.1-4                    AnnotationHub_2.22.1
  [3] BiocFileCache_1.14.0          NMF_0.23.0
  [5] plyr_1.8.6                    igraph_1.2.6
  [7] RCy3_2.10.2                   lazyeval_0.2.2
  [9] splines_4.0.5                 entropy_1.3.0
 [11] BiocParallel_1.24.1           scater_1.18.6
 [13] rncl_0.8.4                    ggplot2_3.3.3
 [15] gridBase_0.4-7                digest_0.6.27
 [17] foreach_1.5.1                 htmltools_0.5.1.1
 [19] viridis_0.6.1                 fansi_0.4.2
 [21] magrittr_2.0.1                memoise_2.0.0
 [23] cluster_2.1.1                 doParallel_1.0.16
 [25] ROCR_1.0-11                   limma_3.46.0
 [27] annotate_1.68.0               R.utils_2.10.1
 [29] prettyunits_1.1.1             colorspace_2.0-1
 [31] blob_1.2.1                    rappdirs_0.3.3
 [33] xfun_0.23                     dplyr_1.0.6
 [35] crayon_1.4.1                  RCurl_1.98-1.3
 [37] bigmemory.sri_0.1.3           graph_1.68.0
 [39] genefilter_1.72.1             phylobase_0.8.10
 [41] survival_3.2-10               iterators_1.0.13
 [43] ape_5.5                       glue_1.4.2
 [45] registry_0.5-1                gtable_0.3.0
 [47] zlibbioc_1.36.0               XVector_0.30.0
 [49] DelayedArray_0.16.3           BiocSingular_1.6.0
 [51] kernlab_0.9-29                Rhdf5lib_1.12.1
 [53] shape_1.4.6                   SingleCellExperiment_1.12.0
 [55] HDF5Array_1.18.1              scales_1.1.1
 [57] DBI_1.1.1                     edgeR_3.32.1
 [59] rngtools_1.5                  Rcpp_1.0.6
 [61] viridisLite_0.4.0             xtable_1.8-4
 [63] progress_1.2.2                rsvd_1.0.5
 [65] bit_4.0.4                     glmnet_4.1-1
 [67] netSmooth_1.10.0              httr_1.4.2
 [69] RColorBrewer_1.1-2            ellipsis_0.3.2
 [71] scuttle_1.0.4                 pkgconfig_2.0.3
 [73] XML_3.99-0.6                  R.methodsS3_1.8.1
 [75] dbplyr_2.1.1                  locfit_1.5-9.4
 [77] utf8_1.2.1                    RJSONIO_1.3-1.4
 [79] howmany_0.3-1                 tidyselect_1.1.1
 [81] rlang_0.4.11                  softImpute_1.4-1
 [83] reshape2_1.4.4                later_1.2.0
 [85] AnnotationDbi_1.52.0          munsell_0.5.0
 [87] BiocVersion_3.12.0            tools_4.0.5
 [89] cachem_1.0.5                  generics_0.1.0
 [91] RSQLite_2.2.7                 ExperimentHub_1.16.1
 [93] ade4_1.7-16                   evaluate_0.14
 [95] stringr_1.4.0                 fastmap_1.1.0
 [97] yaml_2.2.1                    knitr_1.33
 [99] bit64_4.0.5                   purrr_0.3.4
[101] sparseMatrixStats_1.2.1       nlme_3.1-152
[103] mime_0.10                     R.oo_1.24.0
[105] pracma_2.3.3                  xml2_1.3.2
[107] compiler_4.0.5                beeswarm_0.3.1
[109] curl_4.3.1                    interactiveDisplayBase_1.28.0
[111] tibble_3.1.2                  RNeXML_2.4.5
[113] stringi_1.6.2                 RSpectra_0.16-0
[115] lattice_0.20-41               Matrix_1.3-2
[117] vctrs_0.3.8                   pillar_1.6.1
[119] lifecycle_1.0.0               rhdf5filters_1.2.1
[121] BiocManager_1.30.15           combinat_0.0-8
[123] BiocNeighbors_1.8.2           zinbwave_1.12.0
[125] data.table_1.14.0             irlba_2.3.3
[127] bitops_1.0-7                  httpuv_1.6.1
[129] R6_2.5.0                      promises_1.2.0.1
[131] gridExtra_2.3                 vipor_0.4.5
[133] codetools_0.2-18              MASS_7.3-53.1
[135] assertthat_0.2.1              pkgmaker_0.32.2
[137] withr_2.4.2                   GenomeInfoDbData_1.2.4
[139] locfdr_1.1-8                  hms_1.1.0
[141] beachmat_2.6.4                grid_4.0.5
[143] tidyr_1.1.3                   DelayedMatrixStats_1.12.3
[145] rmarkdown_2.8                 Rtsne_0.15
[147] clusterExperiment_2.10.1      shiny_1.6.0
[149] ggbeeswarm_0.6.0              tinytex_0.31



--

*Shraddha Pai, PhD*
Principal Investigator
shraddhapai.com; @spaiglass on Twitter


*Ontario Institute for Cancer Research*
MaRS Centre, 661 University Avenue, Suite 510, Toronto, Ontario, Canada M5G
0A3
*@OICR_news* <https://twitter.com/oicr_news> | *www.oicr.on.ca*
<http://www.oicr.on.ca/>



*Collaborate. Translate. Change lives.*



This message and any attachments may contain confidentia...{{dropped:11}}
#
You are behind the release and should update.  However, with a reasonably
close
instance I can't reproduce your error.  I had a flaky download at one
point; you might
want to reconstitute your brca data.
removing 11654 sampleMap rows not in names(experiments)
  removing 1 colData rownames not in sampleMap 'primary'> brcaA
MultiAssayExperiment object of 4 listed
 experiments with user-defined names and respective classes.
 Containing an ExperimentList class object of length 4:
 [1] BRCA_miRNASeqGene-20160128: SummarizedExperiment with 1046 rows
and 849 columns
 [2] BRCA_mRNAArray-20160128: SummarizedExperiment with 17814 rows and
590 columns
 [3] BRCA_RPPAArray-20160128: SummarizedExperiment with 226 rows and 937 columns
 [4] BRCA_Methylation_methyl27-20160128: SummarizedExperiment with
27578 rows and 343 columns
Functionality:
 experiments() - obtain the ExperimentList instance
 colData() - the primary/phenotype DataFrame
 sampleMap() - the sample coordination DataFrame
 `$`, `[`, `[[` - extract colData columns, subset, or experiment
 *Format() - convert into a long or wide DataFrame
 assays() - convert ExperimentList to a SimpleList of matrices
 exportClass() - save all data to files>
assay(experiments(brca)[[4]])<27578 x 343> matrix of class
DelayedMatrix and type "double":
           TCGA-A1-A0SD-01A-11D-A112-05 ... TCGA-E2-A15T-01A-11D-A112-05
cg00000292                   0.60555263   .                   0.25951246
cg00002426                   0.06197412   .                   0.04144839
cg00003994                   0.33345006   .                   0.34969161
cg00005847                   0.67217034   .                   0.83666264
cg00006414                           NA   .                           NA
       ...                            .   .                            .
cg27657283                   0.04196141   .                   0.03385843
cg27661264                   0.51756951   .                   0.47593756
cg27662379                   0.01078112   .                   0.01157093
cg27662877                   0.03112835   .                   0.02603354
cg27665659                   0.01072116   .
0.01756740> sessionInfo()R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
LC_MESSAGES=C
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
LC_ADDRESS=C
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils
datasets  methods   base

other attached packages:
 [1] rhdf5_2.34.0                curatedTCGAData_1.12.1
MultiAssayExperiment_1.16.0
 [4] SummarizedExperiment_1.20.0 Biobase_2.50.0
GenomicRanges_1.42.0
 [7] GenomeInfoDb_1.26.7         IRanges_2.24.1
S4Vectors_0.28.1
[10] BiocGenerics_0.36.1         MatrixGenerics_1.2.1
matrixStats_0.59.0
[13] httr_1.4.2                  magrittr_2.0.1
AnVIL_1.5.0
[16] dplyr_1.0.6                 BiocManager_1.30.15

loaded via a namespace (and not attached):
 [1] bitops_1.0-7                  bit64_4.0.5
 [3] rprojroot_2.0.2               tools_4.0.3
 [5] R6_2.5.0                      HDF5Array_1.18.1
 [7] DBI_1.1.1                     colorspace_2.0-1
 [9] rhdf5filters_1.2.1            withr_2.4.2
[11] tidyselect_1.1.1              prettyunits_1.1.1
[13] processx_3.5.2                bit_4.0.4
[15] curl_4.3.1                    compiler_4.0.3
[17] cli_2.5.0                     formatR_1.11
[19] DelayedArray_0.16.3           scales_1.1.1
[21] callr_3.7.0                   rappdirs_0.3.3
[23] rapiclient_0.1.3              digest_0.6.27
[25] XVector_0.30.0                pkgconfig_2.0.3
[27] htmltools_0.5.1.1             dbplyr_2.1.1
[29] fastmap_1.1.0                 rlang_0.4.11
[31] rstudioapi_0.13               RSQLite_2.2.7
[33] shiny_1.6.0                   generics_0.1.0
[35] jsonlite_1.7.2                RCurl_1.98-1.3
[37] GenomeInfoDbData_1.2.4        futile.logger_1.4.3
[39] Matrix_1.3-2                  Rcpp_1.0.6
[41] munsell_0.5.0                 Rhdf5lib_1.12.1
[43] lifecycle_1.0.0               yaml_2.2.1
[45] zlibbioc_1.36.0               pkgbuild_1.2.0
[47] BiocFileCache_1.14.0          AnnotationHub_2.22.1
[49] grid_4.0.3                    blob_1.2.1
[51] promises_1.2.0.1              ExperimentHub_1.16.1
[53] crayon_1.4.1                  lattice_0.20-41
[55] ps_1.6.0                      pillar_1.6.1
[57] futile.options_1.0.1          glue_1.4.2
[59] BiocVersion_3.12.0            lambda.r_1.2.4
[61] remotes_2.3.0                 vctrs_0.3.8
[63] httpuv_1.6.1                  gtable_0.3.0
[65] purrr_0.3.4                   tidyr_1.1.3
[67] assertthat_0.2.1              cachem_1.0.5
[69] ggplot2_3.3.3                 xfun_0.23
[71] mime_0.10                     xtable_1.8-4
[73] later_1.2.0                   tibble_3.1.2
[75] tinytex_0.32                  AnnotationDbi_1.52.0
[77] memoise_2.0.0                 ellipsis_0.3.2
[79] interactiveDisplayBase_1.28.0


On Wed, Jun 2, 2021 at 3:37 PM Shraddha Pai <shraddha.pai at utoronto.ca>
wrote:
#
Hi Vincent,
Thank you. This is for a workshop with pre-selected versions of R/BioC, so
I have no control over the release.
Will try redownloading the data. I just found it puzzling that the
identical code works in command-line R (ie started from bash shell) but not
in Rstudio but perhaps they have different ways of caching loaded MAE
objects?
Best,Shraddha

On Wed, Jun 2, 2021 at 4:28 PM Vincent Carey <stvjc at channing.harvard.edu>
wrote:
#
On Thu, Jun 3, 2021 at 9:11 AM Shraddha Pai <shraddha.pai at utoronto.ca>
wrote:
I neglected to mention that my transcript was developed in RStudio on
Terra, with RStudio 1.4.1106, 2389bc24

  
    
#
Hi Shraddha,

I am also unable to reproduce this in RStudio Server Version 1.4.1106 using
our Bioc 3.12 Docker container.

There may be some inadvertent version mixing based on the error.
Please make sure the `BiocManager::valid()` is returning `TRUE`.

---
 > brca <- curatedTCGAData::curatedTCGAData("BRCA", c("miRNASeqGene", 
"mRNAArray", "RPPAArray", "Methylation_methyl27"), dry.run=FALSE)
 > brca
A MultiAssayExperiment object of 4 listed
 ?experiments with user-defined names and respective classes.
 ?Containing an ExperimentList class object of length 4:
 ?[1] BRCA_miRNASeqGene-20160128: SummarizedExperiment with 1046 rows 
and 849 columns
 ?[2] BRCA_mRNAArray-20160128: SummarizedExperiment with 17814 rows and 
590 columns
 ?[3] BRCA_RPPAArray-20160128: SummarizedExperiment with 226 rows and 
937 columns
 ?[4] BRCA_Methylation_methyl27-20160128: SummarizedExperiment with 
27578 rows and 343 columns
Functionality:
 ?experiments() - obtain the ExperimentList instance
 ?colData() - the primary/phenotype DataFrame
 ?sampleMap() - the sample coordination DataFrame
 ?`$`, `[`, `[[` - extract colData columns, subset, or experiment
 ?*Format() - convert into a long or wide DataFrame
 ?assays() - convert ExperimentList to a SimpleList of matrices
 ?exportClass() - save all data to files
 > assays(experiments(brca)[[4]])
List of length 1
names(1): counts
 > sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS/LAPACK: 
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so

locale:
 ?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C LC_TIME=en_US.UTF-8
 ?[4] LC_COLLATE=en_US.UTF-8???? LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
 ?[7] LC_PAPER=en_US.UTF-8?????? LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C???????????? LC_MEASUREMENT=en_US.UTF-8 
LC_IDENTIFICATION=C

attached base packages:
[1] parallel? stats4??? stats???? graphics? grDevices utils datasets? 
methods?? base

other attached packages:
 ?[1] rhdf5_2.34.0??????????????? curatedTCGAData_1.12.1 
MultiAssayExperiment_1.16.0
 ?[4] SummarizedExperiment_1.20.0 Biobase_2.50.0 GenomicRanges_1.42.0
 ?[7] GenomeInfoDb_1.26.7???????? IRanges_2.24.1 S4Vectors_0.28.1
[10] BiocGenerics_0.36.1???????? MatrixGenerics_1.2.1 matrixStats_0.59.0

loaded via a namespace (and not attached):
 ?[1] Rcpp_1.0.6??????????????????? lattice_0.20-41 assertthat_0.2.1
 ?[4] digest_0.6.27???????????????? utf8_1.2.1 mime_0.10
 ?[7] BiocFileCache_1.14.0????????? R6_2.5.0 RSQLite_2.2.7
[10] httr_1.4.2??????????????????? pillar_1.6.1 zlibbioc_1.36.0
[13] rlang_0.4.11????????????????? curl_4.3.1 blob_1.2.1
[16] Matrix_1.3-2????????????????? AnnotationHub_2.22.1 RCurl_1.98-1.3
[19] bit_4.0.4???????????????????? HDF5Array_1.18.1 shiny_1.6.0
[22] DelayedArray_0.16.3?????????? compiler_4.0.3 httpuv_1.6.1
[25] pkgconfig_2.0.3?????????????? htmltools_0.5.1.1 tidyselect_1.1.1
[28] tibble_3.1.2????????????????? GenomeInfoDbData_1.2.4 
interactiveDisplayBase_1.28.0
[31] fansi_0.5.0?????????????????? withr_2.4.2 crayon_1.4.1
[34] dplyr_1.0.6?????????????????? dbplyr_2.1.1 later_1.2.0
[37] rhdf5filters_1.2.1??????????? bitops_1.0-7 rappdirs_0.3.3
[40] grid_4.0.3??????????????????? xtable_1.8-4 lifecycle_1.0.0
[43] DBI_1.1.1???????????????????? magrittr_2.0.1 cachem_1.0.5
[46] XVector_0.30.0??????????????? promises_1.2.0.1 ellipsis_0.3.2
[49] generics_0.1.0??????????????? vctrs_0.3.8 Rhdf5lib_1.12.1
[52] tools_4.0.3?????????????????? bit64_4.0.5 glue_1.4.2
[55] purrr_0.3.4?????????????????? BiocVersion_3.12.0 fastmap_1.1.0
[58] yaml_2.2.1??????????????????? AnnotationDbi_1.52.0 
ExperimentHub_1.16.1
[61] BiocManager_1.30.15?????????? memoise_2.0.0
---

Best,

Marcel
On 6/3/21 11:21 AM, Vincent Carey wrote:
---
Marcel Ramos
Bioconductor Core Team
Roswell Park Comprehensive Cancer Center
Dept. of Biostatistics & Bioinformatics
Elm St. & Carlton St.
Buffalo, New York 14263



This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.