[Bioc-devel] "length(url) == 1 is not TRUE" error for vignette only on build system

bfc <- BiocFileCache()
path <- bfcrpath(bfc, url)

To follow up on this:

Please check the package code for adding/accessing resources in the netDx
package.  If the BiocFileCache options were used correctly there should not
have been multiple entries added to the builders cache as Martin
demonstrated below.

Once the package has been updated to ensure duplicate entries will not be
added, we can clean up the builders.  The package code should be adjusted
first to ensure that users will not encounter the same situation.


Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263
------------------------------
*From:* Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of Martin
Morgan <mtmorgan.bioc at gmail.com>
*Sent:* Friday, September 11, 2020 6:22 PM
*To:* Pages, Herve <hpages at fredhutch.org>; Shraddha Pai <
shraddha.pai at utoronto.ca>
*Cc:* bioc-devel at r-project.org <Bioc-devel at r-project.org>
*Subject:* Re: [Bioc-devel] "length(url) == 1 is not TRUE" error for
vignette only on build system

bfcquery() just searches the cache, so if you've created two resources
that match the name then you end up with two rids

xx = bfcnew(bfc, "foo")
bfcquery(bfc, "foo", "rname")

# A tibble: 1 x 10
  rid   rname create_time access_time rpath rtype fpath last_modified_t?
etag
  <chr> <chr> <chr>       <chr>       <chr> <chr> <chr>            <dbl>
<chr>
1 BFC3  foo   2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
# ? with 1 more variable: expires <dbl>

xx = bfcnew(bfc, "foo_bar")
bfcquery(bfc, "foo", "rname")

# A tibble: 2 x 10
  rid   rname create_time access_time rpath rtype fpath last_modified_t?
etag
  <chr> <chr> <chr>       <chr>       <chr> <chr> <chr>            <dbl>
<chr>
1 BFC3  foo   2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
2 BFC4  foo_? 2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
# ? with 1 more variable: expires <dbl>

and even

xx = bfcnew(bfc, "foo_bar")
bfcquery(bfc, "foo", "rname")

# A tibble: 3 x 10
  rid   rname create_time access_time rpath rtype fpath last_modified_t?
etag
  <chr> <chr> <chr>       <chr>       <chr> <chr> <chr>            <dbl>
<chr>
1 BFC3  foo   2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
2 BFC4  foo_? 2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
3 BFC5  foo_? 2020-09-11? 2020-09-11? /Use? rela? 64ea?               NA NA
# ? with 1 more variable: expires <dbl>

If the cache is under netDx control, then you can be careful about
creating new resources by checking first whether the resource exists, as
outlined in
http://secure-web.cisco.com/1t9-aFwD7wK-0S-MuqPB5uIzT5hQ2gyAD8KMGooS3HThd0k45RZ-frPKl-6Yl2EwZFFcmutTlgEODe0vTjmGHdvrtnhcf5beUQTSS3KiUAl4eWVWv4m3kjsTCUxTVQ2Ll5cGeus1QPlv2osdgHIycOGo6WKDjPqb6PoCxqWOMpEB_x8cSenUxF2N0OkDSEKuwlgJzHwqGjVOwqZQ_odXuqKEnKXz5moLlJO0XSz5XXkTj4kTGwtMAjqKAk_tR8DujPGfYr6Cc70ReKMZUWNVIOc4JgCOAmkILBW94Rp0x_3CX-8DEDeaRhlrgoM7MxkqHGyB0UxHyCXGepxXY196Uglz7WmH1uY02-XQsQ_6e8og/http%3A%2F%2Fbioconductor.org%2Fpackages%2Fdevel%2Fbioc%2Fvignettes%2FBiocFileCache%2Finst%2Fdoc%2FBiocFileCache.html%23cache-to-manage-package-data

If one is managing internet resources, just  use bfcadd

bfc <- BiocFileCache()
path <- bfcrpath(bfc, url)

adding rname '
https://secure-web.cisco.com/1lUwUbgjSoXsv6f7JfdXhRG2f7_ISpNGSvQkNDjD3wQSEEMkTAVHfopgsowEo1GgsXrbByzyoKcAqYa3Kw2bxocuFRTwCIF5UxwWY7fw_FCeNByJcX4L4A9pGyPD1Il4OWJRoYewqI9SOxbFsK6XqqYQmOxhzALdAwDrJw1cOOpppzIIoz5unFJOO2Ihuca8QuIswQZGQqxSBXbeuu8WxSl9QoenolmV7PHovZu_sUMd1DYSflBtliuDz8hRJHSdp-N4wLsvl5GZXxTcJH4ZruhecvOJPR7aGrHCRLZln3wWoMKctL_Oc2xnUYu0ZYrCrVD5MQ8Wf7fgFEl7RPTt5RbTMxrbLO0zXxZZaEwhL5AU/https%3A%2F%2Fbioconductor.org%2Findex.html
'
  |======================================================================|
100%

path <- bfcrpath(bfc, url)

where the first time 'url' is used (as a unique key) the resource is
downloaded to the cache; the second time it is simply accessed from the
cache.

It might be necessary to manually 'clean up' the builders

Martin


On 9/11/20, 4:10 PM, "Bioc-devel on behalf of Pages, Herve" <
bioc-devel-bounces at r-project.org on behalf of hpages at fredhutch.org> wrote:

    I'm not a BiocFileCache expert, sorry. You will probably get better
help
    by opening a BiocFileCache issue on GitHub.

    Cheers,
    H.


    On 9/11/20 12:00, Shraddha Pai wrote:

    > Hi Herve,
    > Sorry - hit shortcut to "send" by mistake.
    >
    > This is the code snippet that got cut off:
    > if (length(rid)>1) {
    > x <- as.POSIXct(rid_rec$last_modified_time)
    > y <- order(x,decreasing=TRUE)
    > rid <- rid[y[1]]
    > }
    >
    > Overkill? I could just take rid[1] but that would not be methodical.
    >
    > Will file issue once you confirm that the problem is that there

should

    > be a unique rid associated with each rname.
    >
    > Thanks again - appreciate it,
    > Shraddha
    >
    > On Fri, Sep 11, 2020 at 2:57 PM Shraddha Pai <

shraddha.pai at utoronto.ca

    > <mailto:shraddha.pai at utoronto.ca <shraddha.pai at utoronto.ca>>> wrote:
    >
    >     Hi Herve,
    >     Thank you for pinpointing this.
    >     Just so I'm clear: rid should not have two IDs correct?
    >     For each rname, the cache should only have a unique ID, right?

Or is

    >     that not the case?
    >
    >     To get the package to build I will take the rid with the latest
    >     "last_modified_time".
    >     Looks like this can be achieved with:
    >     if (length(rid)>1) {
    >     x <- rid_rec$last_modified
    >
    >     On Fri, Sep 11, 2020 at 1:22 PM Pages, Herve <

hpages at fredhutch.org

    >     <mailto:hpages at fredhutch.org <hpages at fredhutch.org>>> wrote:
    >
    >         Hi Shraddha,
    >
    >         Seems to be a BiocFileCache issue.
    >
    >         On my laptop the following code (taken from your
    >         Predict_CaseControl_from_CNV.Rnw vignette):
    >
    >             require(BiocFileCache)
    >             geneURL <- paste("

http://secure-web.cisco.com/1q6dletAEEmECRlpAh7w_w4AmdNAhzos1ijnImWIdzsuXHz3kMIav7Urj3RzYO0S8BzLMKG_0Zcp4ytMlPfA9Ht6IFyJIoCQY-9mRDq-BQDDX_zer8B-bkuIy0hvpOFRkd4a0lSKnviwzssrArVnDcHSCh7EQZsCSMgdEeFsojc3WO3wt8cG5GJztsLKILl9NFGnr2g19zjUcuvqtOBAyec62CVmCfonqPnOWy_VEUPd4WcDd2XwFUW7vEzrFcpC3V6K5pAc0bWjjqEz-3RLU0cWFYgNOGa7zIBwHCX8XMu1JcpLCVoXB-W_RXWMiOwtaHwy8vGei3fBckztfRFgZnA/http%3A%2F%2Fdownload.baderlab.org%2FnetDx%2F

    >         <

https://secure-web.cisco.com/1pBAaf568Sh8T9WwtZnai1icCdNVYUY8FRTQfjYtlK3gI5radqfCsKfzfJz-0BFk_9sTpehBBo60575Jx0MEnZeJRJIKZxiwx2eQ0X90O_CRwVziGst9BfByxNmW5LhvoYTWWSYm1G7ILPw13lka_Q4K2MiNJUX9s0gU6JDcCjBZ6000t2wqGJ_YEBcAGdt30Uv9ElTiYcAXps8tQzLNc9DGwx8ErTvnVmMjvbBVbBRQEBe3Gvo7OcE4-TS_nIEzsZhyvsT1V8xJ3KU9bTb_0J2S1AWC3ETe48ipc0WcJ_cA0OkCYMR7v5dIhE58VGsox/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__download.baderlab.org_netDx_%26d%3DDwMFaQ%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DV-k2jGkEksU1mmcALiO9azNiQrGrVrfwlL4vMvJZwHQ%26s%3DDG1UzWYSS8kbCIEYszKms6FBBQ-GZKWplwuOOOIf88c%26e%3D

",
    >                     "supporting_data/refGene.hg18.bed",sep="")
    >             cache <- rappdirs::user_cache_dir(appname = "netDx")
    >             bfc <- BiocFileCache::BiocFileCache(cache,ask=FALSE)
    >             rid_rec <- bfcquery(bfc, "hg18_genes", "rname")
    >             rid <- rid_rec$rid
    >
    >         returns only one rid:
    >

    >             > rid

    >             [1] "BFC4"
    >
    >         whereas on malbec1 it returns two:
    >

    >             > rid

    >             [1] "BFC6"  "BFC11"
    >
    >         In both cases, I'm using the latest version of BiocFileCache
    >         (1.13.1).
    >
    >         I suspect this has to do with the cache being in different
    >         states on the
    >         two machines.
    >
    >         The fact that 'rid' has a length of 2 causes the failure a

few

    >         lines
    >         later when bfcdownload() is called:
    >
    >             rid <- rid_rec$rid
    >             if (!length(rid)) {
    >                   rid <- names(bfcadd(bfc, "hg18_genes", geneURL))
    >             }
    >             if (!isFALSE(bfcneedsupdate(bfc, rid))){
    >                   bfcdownload(bfc, rid,ask=FALSE)
    >             }
    >
    >         I suggest that you open a BiocFileCache issue on GitHub.
    >
    >         Thanks,
    >         H.
    >
    >
    >         On 9/11/20 08:51, Shraddha Pai wrote:

    >          > Hello BioC-devel team,
    >          > Any idea why netDx 1.1.4 fails on the bioc devel build

system?

    >          >
    >          > This is the error:
    >          >
    >          > --- re-building 'Predict_CaseControl_from_CNV.Rmd' using

knitr

    >          > Quitting from lines 173-195

(Predict_CaseControl_from_CNV.Rmd)

    >          > Error: processing vignette

'Predict_CaseControl_from_CNV.Rmd'

    >         failed

    >          > with diagnostics:
    >          > length(url) == 1 is not TRUE
    >          > --- failed re-building 'Predict_CaseControl_from_CNV.Rmd'
    >          >
    >          >
    >          > I just set up the Bioc-devel docker, cloned the repo using
    >          > git clone git at git.bioconductor.org:packages/netDx
    >          > and ran the vignette, and it runs through successfully.
    >          >
    >          > However the build error occurs across all platforms so

    >         something needs to

    >          > change.
    >          >
    >          > Thank you,
    >          > Shraddha
    >          >
    >          >       [[alternative HTML version deleted]]
    >          >
    >          > _______________________________________________
    >          > Bioc-devel at r-project.org <mailto:Bioc-devel at r-project.org

<Bioc-devel at r-project.org>>

    >         mailing list

    >          >

    >

https://secure-web.cisco.com/1Q1gRqlh1AbjlYoYoW-Tao_IKz59ZWyREYyVSz_yAJWP75JqkTLfdvqwGnEwk-lQEa0GIswGLe7jTmscS0aE9zb844BYJvAdTWLUneOW1-5ZUjtTOWpbCs88HMYRlKLIrA9cP4HevAvVxueol9bqn0j0LHw-UZX-k72wAWe5tjjd54VbNWOM5kKAPNuti9OB4nxoSRxmrVXUKKx2oQB0wT_yROTrgBTc1EAjjJ1P3fI85SfYyYA2SwxLeltUGcEkWoxzDXL1y7atMucMCnMJ2fv8KN41qlhh0uuv8VaHXyIXcxcg76La78NT4MVLsfX2D/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel%26d%3DDwICAg%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DzwoVT5i-NLz7FqpRg0pSLIqNJBJiI_Ac8FigI43cwRc%26s%3DYhavBW80Elhyr-R7RuyrA2bwYPfI08HAsraBEs2FRU0%26e%3D

    >          >

    >
    >         --
    >         Herv? Pag?s
    >
    >         Program in Computational Biology
    >         Division of Public Health Sciences
    >         Fred Hutchinson Cancer Research Center
    >         1100 Fairview Ave. N, M1-B514
    >         P.O. Box 19024
    >         Seattle, WA 98109-1024
    >
    >         E-mail: hpages at fredhutch.org <mailto:hpages at fredhutch.org

<hpages at fredhutch.org>>

    >         Phone:  (206) 667-5791
    >         Fax:    (206) 667-1319
    >

    --
    Herv? Pag?s

    Program in Computational Biology
    Division of Public Health Sciences
    Fred Hutchinson Cancer Research Center
    1100 Fairview Ave. N, M1-B514
    P.O. Box 19024
    Seattle, WA 98109-1024

    E-mail: hpages at fredhutch.org
    Phone:  (206) 667-5791
    Fax:    (206) 667-1319