Wolfgang Huber
on Sun, 7 May 2023 14:29:37 +0200 writes:
> Hi Martin As you correctly point out, Bioconductor package
> developers are probably not those with the most relevant
> use cases. I think there are use cases for everyone
> else?anyone who decides to write code on R-devel, for
> whatever reason, and just wants to use a Bioconductor
> package between mid-April to mid-October (they could
> develop for CRAN, or just be a user and write scripts and
> packages for a private project). There are many useful
> packages on Bioconductor that are of general interest,
> even for people whose work does not center around
> Bioconductor or biology (say, ggtree, rhdf5,
> sparseMatrixStats, EBImage, ?)
> Thanks and best wishes Wolfgang
As the older ones among you know, I've been a BioC developer
only many years ago ('hexbin' e.g.), but as an R package
maintainer and co-maintainer and R Core team member,
I really like to chime in here, declaring that it *has* been
quite painful for me over the years to test CRAN packages which
depend on BioC packages - with R-devel -- which is my primary R
version for testing, notably also for testing potential changes in R
across many packages, etc.
Notably during this half of the year where there is no
"official" way how to correctly install current Bioconductor packages
(in their own package library, as I always do) under R-devel.
If I'd be able to sum up the time lost over this issue for the last say 10
years, it would add to a full working day at least. ...
(and I have added a comment also in the above issue #108)
> (PS in my particular case yesterday, it was just that my
> R-devel is better maintained (built from source etc) and
> has in its library some (non-BioC) packages with complex
> systems dependencies that I need for a workflow I am
> working on, packages that currently elude me on my binary
> installation of R4.3. And then in addition I just wanted
> to *use* a package from Bioconductor and didn?t like how
> clumsy that experience was.)
My other experience is that I always have to help people in my
group to install our pcalg CRAN package because it depends
e.g. on Bioc packages 'graph' and 'Rgraphviz' .. and on their
laptops they somehow don't have the correct getOption("repos")
or there are other reasons why install.packages('pcalg')
does not find its Bioc dependencies.
On our Linux desktop+server environment, I do setup
options(repos = ....)
such that everything works .. but alas, also *not* when in
R-devel but when you develop a package for CRAN / or only just
follow the more wide recommendation to also check your package
with current R-devel, then non-expert package developers need a
lot of stamina if their package depends (directly or
recursively) on a Bioc package....
which is really unfortunate and tends to put the Bioconductor
project in a shady light it really has not deserved at all!
Martin
--
Martin Maechler
ETH Zurich and R Core team
>> Il giorno 06.05.2023, alle ore 16:45, Martin Morgan
>> <mtmorgan.bioc at gmail.com> ha scritto:
>>
>> I opened two issues for further discussion of the
>> technical aspects.
>> https://github.com/Bioconductor/BiocManager/issues/165
>> https://github.com/Bioconductor/pkgrevdocs/issues/108
>> Just to be clear, as noted at the end of the second issue
>> and on the web page you mention, a Bioconductor package
>> developer wishing to use 'Bioc-devel' should, during the
>> mid-April to mid-October release cycle, be using the
>> **release** version of R. This combination of R and
>> Bioconductor is supported by BiocManager. Similarly, in
>> the mid-October to mid-April release cycle, the
>> Bioconductor developer should be R-devel, and BoicManager
>> supports this, too. There are scenarios where a
>> developer might wish to combine R-devel and Bioc-devel in
>> the mid-May, to mid-October time frame, e.g., when
>> developing a CRAN package with Bioconductor dependencies,
>> or when conscientiously testing CRAN packages that depend
>> on Bioconductor packages. One may also just want to be on
>> the bleeding edge, so using R-devel and living with any
>> consequence that arise from R / Bioconductor version
>> mismatches. Are these less-common scenarios the one that
>> you are engaged in? Martin From: Bioc-devel
>> <bioc-devel-bounces at r-project.org> on behalf of Wolfgang
>> Huber <wolfgang.huber at embl.org> Date: Saturday, May 6,
>> 2023 at 9:43 AM To: Vincent Carey
>> <stvjc at channing.harvard.edu> Cc: bioc-devel at r-project.org
>> <bioc-devel at r-project.org> Subject: Re: [Bioc-devel]
>> BiocManager::install Dear Martin and Vince
>>
>> thank you, very insightful points. Indeed I think it?s
>> primarily a matter of documentation and priming, and,
>> e.g., adding Martin's lines prominently enough e.g. to
>> https://contributions.bioconductor.org/use-devel.htmland
>> a reference to it into the manpage of
>> BiocMananger::install.
>>
>> I acknowledge that installation and dealing with
>> dependencies is *hard*. The relatively smooth user
>> experience of Bioconductor, compared to other projects,
>> is one of our greatest assets. I guess it needs constant
>> attention on our side. One of the slogans of
>> R/Bioconductor is ?turning users into developers? and
>> therefore something that has useful defaults but is easy
>> enough to customize seems desirable. In that sense, it?d
>> be great to be able to stay with BiocManager::install and
>> not having to abandon it in favour of
>> base::install.packages.
>>
>> The codebase behind BiocManager::install seems to have
>> become a little?. complicated.
>>
>> The documentation clarification re
>> BIOCONDUCTOR_ONLINE_VERSION_DIAGNOSIS that Martin
>> suggests would be welcome.
>>
>> Kind regards Wolfgang
>>
>>
>>
>>
>>
>>
>> > Il giorno 06.05.2023, alle ore 13:05, Vincent Carey
>> <stvjc at channing.harvard.edu> ha scritto:
>> >
>> > Thanks for these observations Wolfgang, I am glad I
>> read to the end, > because as you say,
>> have no > experience with renv or Connect > much of the
>> motivating detail is opaque to me.
>> >
>> > I would question the proposition
>> >
>> > "Given the structural differences between BioConductor
>> and CRAN > repositories, it is not straightforward to
>> work with both types. "
>> >
>> > with at least 10 years of history of effective usage of
>> both together > by many hundreds of users.
>> "Straightforward" is > subjective. The existence of some
>> shortcomings, like the specific > ones you mention, is
>> acknowledged, and setting > up priorities to ameliorate
>> them would be worthwhile. Part of the > prioritization
>> would need to be based on user > data and user
>> experiences. In the case of this posit.co article, what
>> > is known about the significance of Connect > for
>> genomic data science? I have not had great difficulty
>> publishing > apps to shinyapps.io that use Bioconductor >
>> and CRAN, but perhaps it can be made easier if that is a
>> key concern.
>> >
>> > The problem of smoothly supporting multiple versions of
>> R/Bioc > simultaneously is also acknowledged. At this >
>> time we do not have sufficient resources to make a big
>> charge in the > direction of increasing support for this
>> > "use case". Users and sysadmins with sufficient
>> expertise can > definitely accomplish much in this area,
>> see >
>> https://bioconductor.org/about/release-announcements/ for
>> the map of > resources supporting this going back to >
>> 2005. If there is a way to simplify this by using
>> recently developed > package management strategies is
>> would > be good to know and document.
>> >
>> > This is a good place to continue the discussion from a
>> developer's > perspective, but how can we get more >
>> input from non-developer users? And from posit.co?
>> >
>> > "Publishing Shiny Apps that make use of BioConductor
>> packages to > Connect is not possible for this setup. >
>> BiocManager::install() temporarily adds the BioConductor
>> repository > for the duration of the install process. >
>> During the publishing process rsconnect no longer has any
>> knowledge > about BioConductor." -- is this something >
>> that can be remedied in BiocManager? Are we able to test
>> Connect for > this use case?
>> >
>> >
>> > On Sat, May 6, 2023 at 4:40?AM Wolfgang Huber
>> <wolfgang.huber at embl.org> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I am wondering whether: >> 1. it could be easier to
>> install Bioconductor packages (devel or release) on
>> R-devel (or other non-standard R versions) using
>> BiocManager::install (I may be stirring a hornet?s nest
>> with that:) >> 2. whether its documentation needs to be
>> updated and/or its implementation could be deconvoluted
>> (hopefully that?s uncontroversial).
>> >>
>> >> Re the first point, I appreciate that we?re trying to
>> help non-expert users with simple use cases, and that we
>> had/have a lot of trouble with users working with
>> out-of-sync versions. OTOH, the current solution (rigid,
>> confusing documentation, seemingly buggy implementation)
>> seems to be standing in the way for developers, a
>> dichotomy that we do not really want.
>> >>
>> >> Of course, a workaround is >> ```{r} >>>
>> install.packages("ggtree", repos = c(?@CRAN@",
>> "https://bioconductor.org/packages/3.18/bioc") >> ``` >>
>> and maybe this is just the answer. So far, my workflows
>> have been based on BiocManager::install, but I get (and
>> cannot seem to get rid of):
>> options(BIOCONDUCTOR_ONLINE_VERSION_DIAGNOSIS = FALSE)
>> >>> BiocManager::install("ggtree", version = "devel") >>
>> >>> sessionInfo() >> R Under development (unstable)
>> (2023-05-05 r84398) >> Platform: aarch64-apple-darwin20
>> (64-bit) >> Running under: macOS Ventura 13.3.1
>> >>
>> >> Matrix products: default >> BLAS:
/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
/Users/whuber/R.framework/Versions/4.4/Resources/lib/libRlapack.dylib;
>> >>
>> >> locale: >> [1]
>> en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>> >>
>> >> time zone: Europe/Berlin >> tzcode source: internal
>> >>
>> >> attached base packages: >> [1] stats graphics
>> grDevices utils datasets methods base
>> >>
>> >> other attached packages: >> [1] BiocManager_1.30.20
>> >>
>> >> loaded via a namespace (and not attached): >> [1]
>> compiler_4.4.0 tools_4.4.0 rstudioapi_0.14 >> ```
>> >>
>> >> I noted some discussion on this here:
>> https://github.com/Bioconductor/BiocManager/issues/13 but
>> this was 5 years ago. >> It appears that the
>> documentation of BiocManager::install mismatches its
>> implementation, and overall the process for something
>> that's conceptually quite simple seems to have become
>> convoluted.
>> >>
>> >> One of the most helpful documentation resources on
>> >>
>> >> Thanks and best wishes >> Wolfgang
>> >>
>> >> --
>> >> Wolfgang Huber >> EMBL >>
>> >>
>> >> _______________________________________________ >>
>> >
>> > --
>> > The information in this e-mail is intended only for