[Bioc-devel] Old package versions / Bioc archive of package's *.tar.gz releases?

Oh, and the svn/git walking knows about release branches so it will find
hot-patch versions correctly, as well.

~G

On Thu, Oct 5, 2017 at 10:01 PM, Gabe Becker <beckerg4 at gene.com> wrote:

Correct. The actual order of checks is:

   1. did switchr already retrieve that exact version for something else
   and keep it around,
   2. A GRANRepository object if one is specified (don't worry much
   about this one)
   3. the manifest itself (cran and bioc source types are ignored, but
   it would walk SCM if it had git/svn type manifest entry for the package)
   4. cran repo and cran archives,
   5. bioc repositories (all of them in descending order),
   6. bioc git (bioc SVN for the version on CRAN, which appeared to
   still work last time I checked it a week or two ago)

See the code in https://github.com/gmbecker/switchr/blob/master/R/
retrievePkgVersion.R for details.

Best,
~G




On Oct 5, 2017 9:06 PM, "Henrik Bengtsson" <henrik.bengtsson at gmail.com>
wrote:

That's really nice; I didn't know it could do all that.  For my
clarification, when using PkgManifest(..., type = "bioc") it'll search
(i) the CRAN archives, (ii) the Bioconductor repos(es), and then (iii)
the Bioconductor Git repos - is that a correct observation? (I
installed from https://github.com/gmbecker/switchr)

/Henrik




On Thu, Oct 5, 2017 at 4:15 PM, Gabe Becker <becker.gabe at gene.com> wrote:

In point of fact, it looks like IRanges 2.6.0 is an instance of that
weakness, so was probably a bad example. 2.6.1 installs correctly, or

would

in it's native R/base bioc environment... (it fails for me in the

library I

have...)

Also, the version on CRAN uses the bioc SVN, so may not work for recent
versions.

On Thu, Oct 5, 2017 at 3:58 PM, Gabe Becker <beckerg4 at gene.com> wrote:

Henrik et al.,

My switchr package (on CRAN, github at:
http://github.com/gmbecker/switchr,  preprint of the paper here:
https://arxiv.org/abs/1501.02284) can do this.

In fact, installing (cohorts of) old versions of packages is one of

it's

primary purposes. Specifically, it can install old source versions of
packages from CRAN, Bioconductor, and general Git and SVN repos you

tell it

about.

With the caveat that it's a bad idea in the general case to specify an

old

version of one package without specifying versions of its dependencies
(switchr allows you to do this via a manifest, which can be

constructed from

sessionInfo output or guessed in the case of a CRAN package), you can

just

do

man = PkgManifest(name="IRanges", type="bioc")

install_packages("IRanges", man, versions = c(IRanges = "2.6.0"))

And you will successfully completely break your Bioc installation by
installing IRanges 2.6.0 into it. ;-)

Switchr also gives you tools to more easilly maintain multiple

libraries

which contain, for example, different bioc versions in them.

NB: switchr is subject to the caveat Martin pointed out and will fail

to

retrieve a buildable version of the package if said buildable version

is not

the first commit in SCM bearing that version in its DESCRIPTION file.

Hope that helps.

Best,
~G

On Thu, Oct 5, 2017 at 2:21 PM, Martin Morgan
<martin.morgan at roswellpark.org> wrote:

On 10/05/2017 05:14 PM, Henrik Bengtsson wrote:

On Thu, Oct 5, 2017 at 1:46 PM, Martin Morgan
<martin.morgan at roswellpark.org> wrote:

On 10/05/2017 01:50 PM, Henrik Bengtsson wrote:

Is there an easily accessible archive for Bioconductor packages
similar to what is provided on CRAN where you can find all released
versions of a package, e.g.
https://cran.r-project.org/src/contrib/Archive/PSCBS/?

Say I want to access the source code for affy 1.18.0.  Here are the
two approaches I'm aware of and none of them are particularly
appealing to me.  Does anyone know of a better approach?

The only option is to scrape, and that's approximate. One could

build

an
archive

pkg,version,branch,from_svn_rev,to_svn_rev

and then consult that. Packages are supposed to increment the 'z' of
x.y.z,
but I'm sure there are many exceptions. I believe Jim Hester has an

svn

script for this, but I wasn't able to locate it; it would be fast in
git.

Thanks.  About 'z' not being increased.  Does the Bioc build servers
release (a) continuously or (b) only when it detects a version change
x.y.z -> x.y.z+1?  If it does it continuously, then what x.y.z is
installed does matter on when it was downloaded/installed, correct?
On the other hand, if it only builds in when a version bump is
detected, then one can at least narrow it down to a much narrow set

of

x.y.z submits (if multiple exists).

The builder only pushes for upward increments, so a commit without a
'positive' version bump would be built but not pushed to the public
repository. I'm not sure how rigorously this policy was enforced

before,

e.g., 2005.

Of course there are exceptions, e.g., it is occasionally (at most one

or

two times a release cycle) necessary to flush the public repository
entirely, and then whatever is built is pushed. And there is nothing
stopping the user from doing a check-out from svn. Perhaps others

will chime

in with the gory / correct details.

Martin

For your future self, this


https://bioconductor.org/packages/3.5/bioc/src/contrib/Archi

ve/S4Vectors/

provides a hint of a change coming with the next release --

archives of

all
RELEASE package versions, starting in Bioc 3.6. (Kudos to Val for
implementing this)

This is great!  Thanks Val for this.

Thanks

Henrik

Martin

# APPROACH 1: Download from http://bioconductor.org

The best approach I know now is to try to guess the date when this

was

released in order to identify the Bioconductor release version.
Something like this:

1. Guess around 2010.

2. Go to http://bioconductor.org/about/release-announcements/ and

see

what R versions were in use during 2010.  I find R 2.6.x and R

2.7.x.

The Bioc version for those R versions (same URL) are Bioc 2.1 and

Bioc

2.2.  Let's focus on Bioc 2.2 (because I happen to know that is the
one)

3. Following the Bioc 2.2 link on above URL to get to
http://bioconductor.org/packages/2.2/BiocViews.html.

4. Click through, one eventually gets to
http://bioconductor.org/packages/2.2/bioc/html/affy.html

5. The "Source" link points to

http://bioconductor.org/packages/2.2/bioc/src/contrib/affy_1

.18.2.tar.gz

Say I wanted affy 1.16.0 instead and I made the wrong guess in

Step 2,

I can extrapolate from (Bioc 2.2, affy 1.18.x) finding that I

should

go to Bioc 2.1 to find affy 1.16.x (because releases have even

minor

version numbers).   It works, but is a bit tedious if you need to

do

this more than once.

Also, I'm pretty sure I read somewhere that Bioc on archive the

most

recent package version under each release, which means there is no
affy_1.18.0.tar.gz available for download.  Is that correct?


# APPROACH 2: Version control

$ git clone https://git.bioconductor.org/packages/affy
$ cd affy

# Package releases/versions are not tagged
$ git tag
[empty]

# Check Bioc release branches
$ git branch -a
* master
    remotes/origin/HEAD -> origin/master
    remotes/origin/RELEASE_1_0
    remotes/origin/RELEASE_1_0_branch
    remotes/origin/RELEASE_1_4
    remotes/origin/RELEASE_1_4_branch
    remotes/origin/RELEASE_1_5
[...]
    remotes/origin/RELEASE_3_5
    remotes/origin/master

That's back to above game of trying to narrow down which Bioc

release

I should look at.  A similar approach is to look at the commit log:

$ git log DESCRIPTION

commit 35573048255b398f99ff1d3560906b2121912248
Author: Herve Pages <hpages at fhcrc.org>
Date:   Mon Apr 24 19:50:57 2017 +0000

      bump x.y.z versions to odd y after creation of 3_5 branch

      git-svn-id:


file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma

n/Rpacks/affy at 129129

bc3139a8-67e5-0310-9ffc-ced21a209358

commit aa4c2d648658e8c2cca2baf651aea92df55a4392
Author: Herve Pages <hpages at fhcrc.org>
Date:   Mon Apr 24 19:25:24 2017 +0000

      bump x.y.z versions to even y prior to creation of 3_5 branch

      git-svn-id:


file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma

n/Rpacks/affy at 129126

bc3139a8-67e5-0310-9ffc-ced21a209358

[...]

and try to locate affy 1.18.0 by peeking at the DESCRIPTION file
history.

Does anyone know of a better/more automated approach?

Thanks,

Henrik

This email message may contain legally privileged and/or

confidential

information.  If you are not the intended recipient(s), or the

employee

or
agent responsible for the delivery of this message to the intended
recipient(s), you are hereby notified that any disclosure, copying,
distribution, or use of this email message is prohibited.  If you

have

received this message in error, please notify the sender

immediately by

e-mail and delete this email message from your computer. Thank you.

This email message may contain legally privileged

and/or...{{dropped:2}}

--
Gabriel Becker, Ph.D
Associate Scientist
Bioinformatics and Computational Biology
Genentech Research

--
Gabriel Becker, Ph.D
Associate Scientist
Bioinformatics and Computational Biology
Genentech Research

--
Gabriel Becker, Ph.D
Associate Scientist
Bioinformatics and Computational Biology
Genentech Research