[Bioc-devel] Do we have an unpacked archive for BioC, so that we can URL link to individual files ?

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated)?way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/DESCRIPTION

-> Is there anything comparable today ?

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated)?way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D
ESCRIPTION

-> Is there anything comparable today ?

I think the git archive command might do the trick, but it requires
ssh?
(i.e., write) access to the repository

git archive --remote=git at git.bioconductor.org:packages/xcms \
??????master DESCRIPTION | tar -xf -

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

This email message may contain legally privileged and/or confidential
information.??If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message to the
intended recipient(s), you are hereby notified that any disclosure,
copying, distribution, or use of this email message is
prohibited.??If you have received this message in error, please
notify the sender immediately by e-mail and delete this email message
from your computer. Thank you.

Hi,

thanks for the answer, unfortunately
not exactly what I meant. To be more precise:

I want to point Galaxy to download (test) MS data files
from either the faahKO package, or msdata or mtbls2 in BioC.

For that I need an ftp/http/https link to the files,
and I was able to use

https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msdata/inst/cdf/ko15.CDF

(which still works, but is probably outdated)
So is there anything equivalent in the new git setup ?

Yours,
Steffen


On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated)?way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D
ESCRIPTION

-> Is there anything comparable today ?

I think the git archive command might do the trick, but it requires
ssh
(i.e., write) access to the repository

git archive --remote=git at git.bioconductor.org:packages/xcms \
 ??????master DESCRIPTION | tar -xf -

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

This email message may contain legally privileged and/or confidential
information.??If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message to the
intended recipient(s), you are hereby notified that any disclosure,
copying, distribution, or use of this email message is
prohibited.??If you have received this message in error, please
notify the sender immediately by e-mail and delete this email message
from your computer. Thank you.

On 11/19/2017 06:11 PM, Neumann, Steffen wrote:

Hi,

thanks for the answer, unfortunately
not exactly what I meant. To be more precise:

I want to point Galaxy to download (test) MS data files
from either the faahKO package, or msdata or mtbls2 in BioC.

For that I need an ftp/http/https link to the files,
and I was able to use

https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msdata/inst/cdf/ko15.CDF

(which still works, but is probably outdated)
So is there anything equivalent in the new git setup ?

no, not from git.bioconductor.org.

Martin

Yours,
Steffen


On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated) way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D
ESCRIPTION

-> Is there anything comparable today ?

I think the git archive command might do the trick, but it requires
ssh
(i.e., write) access to the repository

git archive --remote=git at git.bioconductor.org:packages/xcms \
       master DESCRIPTION | tar -xf -

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

This email message may contain legally privileged and/or confidential
information.  If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message to the
intended recipient(s), you are hereby notified that any disclosure,
copying, distribution, or use of this email message is
prohibited.  If you have received this message in error, please
notify the sender immediately by e-mail and delete this email message
from your computer. Thank you.

This email message may contain legally privileged and/or...{{dropped:2}}

So, what if there was a MetaBioconductor site and repo similar to the
MetaCRAN project? https://www.r-pkg.org/about

Not sure exactly who maintains it (probably someone at RStudio), but it is
largely automatic I believe, and it provides a full GitHub archive of *all*
the packages on CRAN, which if this were done for Bioconductor would
satisfy what I think Steffen is looking for, as well as lots of other
advantages.

It also has the advantage that it looks like 90% of the work would be done
for Bioconductor with likely very little work required for the Bioconductor
Admins (ideally 100% of work would be done for them).

Just a thought.

-Robert

Robert M Flight, PhD
Bioinformatics Research Associate
Puller of Rabbits from Hats
Research Parasite
Resource Center for Stable Isotope Resolved Metabolomics
Manager, Systems Biology and Omics Integration Journal Club
Markey Cancer Center
CC434 Roach Building
University of Kentucky
Lexington, KY

Twitter: @rmflight
Web: rmflight.github.io
ORCID: http://orcid.org/0000-0001-8141-7788
EM rflight79 at gmail.com
PH 502-509-1827 <(502)%20509-1827>

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. - Ronald Fisher

On Mon, Nov 20, 2017 at 8:51 AM Martin Morgan <
martin.morgan at roswellpark.org> wrote:

On 11/19/2017 06:11 PM, Neumann, Steffen wrote:

Hi,

thanks for the answer, unfortunately
not exactly what I meant. To be more precise:

I want to point Galaxy to download (test) MS data files
from either the faahKO package, or msdata or mtbls2 in BioC.

For that I need an ftp/http/https link to the files,
and I was able to use

https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msdata/inst/cdf/ko15.CDF

(which still works, but is probably outdated)
So is there anything equivalent in the new git setup ?

no, not from git.bioconductor.org.

Martin

Yours,
Steffen


On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated) way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D
ESCRIPTION

-> Is there anything comparable today ?

I think the git archive command might do the trick, but it requires
ssh
(i.e., write) access to the repository

git archive --remote=git at git.bioconductor.org:packages/xcms \
       master DESCRIPTION | tar -xf -

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

This email message may contain legally privileged and/or confidential
information.  If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message to the
intended recipient(s), you are hereby notified that any disclosure,
copying, distribution, or use of this email message is
prohibited.  If you have received this message in error, please
notify the sender immediately by e-mail and delete this email message
from your computer. Thank you.

This email message may contain legally privileged and/or...{{dropped:2}}

So, what if there was a MetaBioconductor site and repo similar to the 
MetaCRAN project? https://www.r-pkg.org/about

Not sure exactly who maintains it (probably someone at RStudio), but it 
is largely automatic I believe, and it provides a full GitHub archive of 
*all* the packages on CRAN, which if this were done for Bioconductor 
would satisfy what I think Steffen is looking for, as well as lots of 
other advantages.

It also has the advantage that it looks like 90% of the work would be 
done for Bioconductor with likely very little work required for the 
Bioconductor Admins (ideally 100% of work would be done for them).

Just a thought.

-Robert

Robert M Flight, PhD
Bioinformatics Research Associate
Puller of Rabbits from Hats
Research Parasite
Resource Center for Stable Isotope Resolved Metabolomics
Manager, Systems Biology and Omics Integration Journal Club
Markey Cancer Center
CC434 Roach Building
University of Kentucky
Lexington, KY

Twitter: @rmflight
Web: rmflight.github.io <http://rmflight.github.io>
ORCID: http://orcid.org/0000-0001-8141-7788
EM rflight79 at gmail.com <mailto:rflight79 at gmail.com>
PH 502-509-1827 <tel:(502)%20509-1827>

To call in the statistician after the experiment is done may be no more 
than asking him to perform a post-mortem examination: he may be able to 
say what the experiment died of. - Ronald Fisher

On Mon, Nov 20, 2017 at 8:51 AM Martin Morgan 
<martin.morgan at roswellpark.org <mailto:martin.morgan at roswellpark.org>> 
wrote:

    On 11/19/2017 06:11 PM, Neumann, Steffen wrote:

     > Hi,
     >
     > thanks for the answer, unfortunately
     > not exactly what I meant. To be more precise:
     >
     > I want to point Galaxy to download (test) MS data files
     > from either the faahKO package, or msdata or mtbls2 in BioC.
     >
     > For that I need an ftp/http/https link to the files,
     > and I was able to use
     >
     >

    https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msdata/inst/cdf/ko15.CDF

     >
     > (which still works, but is probably outdated)
     > So is there anything equivalent in the new git setup ?

    no, not from git.bioconductor.org <http://git.bioconductor.org>.

    Martin

     >
     > Yours,
     > Steffen
     >
     >
     > On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

     >> On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

     >>> Hi,
     >>>
     >>> I would like to have a URL to individual files
     >>> we have in BioC packages. This is useful
     >>> e.g. if I need test data from, say, the msdata package,
     >>> in another context.
     >>>
     >>> In the SVN days, I was able to point directly
     >>> into the SVN repo with readonly:readonly access.
     >>> We also had the (now deprecated)?way to link to:
     >>> https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D
     >>> ESCRIPTION
     >>>
     >>> -> Is there anything comparable today ?

     >>
     >> I think the git archive command might do the trick, but it requires
     >> ssh
     >> (i.e., write) access to the repository
     >>
     >> git archive --remote=git at git.bioconductor.org:packages/xcms \
     >>? ??????master DESCRIPTION | tar -xf -
     >>
     >>

     >>>
     >>> It could be as simple as a directory tree
     >>> of a working copies of all https://git.bioconductor.org/
     >>> served by the apache (or whatever) web server,
     >>> or (a bit cooler) a gitweb or similar http interface.
     >>>
     >>> Wouldn't that be a fantastic christmas present
     >>> for us users and developers :-) ?
     >>>
     >>> Yours,
     >>> Steffen
     >>>

     >>
     >>
     >> This email message may contain legally privileged and/or

    confidential

     >> information.??If you are not the intended recipient(s), or the
     >> employee or agent responsible for the delivery of this message

    to the

     >> intended recipient(s), you are hereby notified that any disclosure,
     >> copying, distribution, or use of this email message is
     >> prohibited.??If you have received this message in error, please
     >> notify the sender immediately by e-mail and delete this email

    message

     >> from your computer. Thank you.

    This email message may contain legally privileged and/or...{{dropped:2}}

On 11/20/2017 04:58 PM, Robert M. Flight wrote:

So, what if there was a MetaBioconductor site and repo similar to the
MetaCRAN project? https://www.r-pkg.org/about

Not sure exactly who maintains it (probably someone at RStudio), but it
is largely automatic I believe, and it provides a full GitHub archive of
*all* the packages on CRAN, which if this were done for Bioconductor
would satisfy what I think Steffen is looking for, as well as lots of
other advantages.

It also has the advantage that it looks like 90% of the work would be
done for Bioconductor with likely very little work required for the
Bioconductor Admins (ideally 100% of work would be done for them).

there has been some discussion of that, e.g.,

   https://stat.ethz.ch/pipermail/bioc-devel/2017-September/011455.html

but we will not do it (maintain a github mirror), for instance because
some experiment data packages have commits that would require lfs,
because of issues with reliably  syncing data, and because of confusion
it introduces on the 'official' repository.

We do intend to provide a browsable interface to the official
git.bioconductor.org that would allow Steffen's use case during the
present release cycle; it is not currently available.

A different solution would be to update the relevant ExperimentData
package to use ExperimentHub (see, e.g.,
https://github.com/LTLA/TENxBrainData for a relatively simple example,
though not yet in Bioconductor) and the resource would then be available
at a URL https://annotationhub.bioconductor.org/fetch/...

Martin

Just a thought.

-Robert

Robert M Flight, PhD
Bioinformatics Research Associate
Puller of Rabbits from Hats
Research Parasite
Resource Center for Stable Isotope Resolved Metabolomics
Manager, Systems Biology and Omics Integration Journal Club
Markey Cancer Center
CC434 Roach Building
University of Kentucky
Lexington, KY

Twitter: @rmflight
Web: rmflight.github.io <http://rmflight.github.io>
ORCID: http://orcid.org/0000-0001-8141-7788
EM rflight79 at gmail.com <mailto:rflight79 at gmail.com>
PH 502-509-1827 <tel:(502)%20509-1827>

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to
say what the experiment died of. - Ronald Fisher

On Mon, Nov 20, 2017 at 8:51 AM Martin Morgan
<martin.morgan at roswellpark.org <mailto:martin.morgan at roswellpark.org>>
wrote:

    On 11/19/2017 06:11 PM, Neumann, Steffen wrote:

     > Hi,
     >
     > thanks for the answer, unfortunately
     > not exactly what I meant. To be more precise:
     >
     > I want to point Galaxy to download (test) MS data files
     > from either the faahKO package, or msdata or mtbls2 in BioC.
     >
     > For that I need an ftp/http/https link to the files,
     > and I was able to use
     >
     >

https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msdata/inst/cdf/ko15.CDF

     >
     > (which still works, but is probably outdated)
     > So is there anything equivalent in the new git setup ?

    no, not from git.bioconductor.org <http://git.bioconductor.org>.

    Martin

     >
     > Yours,
     > Steffen
     >
     >
     > On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

     >> On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

     >>> Hi,
     >>>
     >>> I would like to have a URL to individual files
     >>> we have in BioC packages. This is useful
     >>> e.g. if I need test data from, say, the msdata package,
     >>> in another context.
     >>>
     >>> In the SVN days, I was able to point directly
     >>> into the SVN repo with readonly:readonly access.
     >>> We also had the (now deprecated) way to link to:
     >>>

https://raw.githubusercontent.com/Bioconductor-mirror/xcms/master/D

     >>> ESCRIPTION
     >>>
     >>> -> Is there anything comparable today ?

     >>
     >> I think the git archive command might do the trick, but it

requires

     >> ssh
     >> (i.e., write) access to the repository
     >>
     >> git archive --remote=git at git.bioconductor.org:packages/xcms \
     >>        master DESCRIPTION | tar -xf -
     >>
     >>

     >>>
     >>> It could be as simple as a directory tree
     >>> of a working copies of all https://git.bioconductor.org/
     >>> served by the apache (or whatever) web server,
     >>> or (a bit cooler) a gitweb or similar http interface.
     >>>
     >>> Wouldn't that be a fantastic christmas present
     >>> for us users and developers :-) ?
     >>>
     >>> Yours,
     >>> Steffen
     >>>

     >>
     >>
     >> This email message may contain legally privileged and/or

    confidential

     >> information.  If you are not the intended recipient(s), or the
     >> employee or agent responsible for the delivery of this message

    to the

     >> intended recipient(s), you are hereby notified that any

disclosure,

     >> copying, distribution, or use of this email message is
     >> prohibited.  If you have received this message in error, please
     >> notify the sender immediately by e-mail and delete this email

    message

     >> from your computer. Thank you.

    This email message may contain legally privileged

and/or...{{dropped:2}}

list

This email message may contain legally privileged and/or confidential
information.  If you are not the intended recipient(s), or the employee or
agent responsible for the delivery of this message to the intended
recipient(s), you are hereby notified that any disclosure, copying,
distribution, or use of this email message is prohibited.  If you have
received this message in error, please notify the sender immediately by
e-mail and delete this email message from your computer. Thank you.

The other thing Steffen could do is keep personal clones of those
packages on GitHub, but that's probably not ideal.

-Robert

On Mon, Nov 20, 2017 at 4:58 PM Robert M. Flight <rflight79 at gmail.com

wrote:
So, what if there was a MetaBioconductor site and repo similar to
the MetaCRAN project??https://www.r-pkg.org/about

Not sure exactly who maintains it (probably someone at RStudio),
but it is largely automatic I believe, and it provides a full
GitHub archive of *all* the packages on CRAN, which if this were
done for Bioconductor would satisfy what I think Steffen is looking
for, as well as lots of other advantages.?

It also has the advantage that it looks like 90% of the work would
be done for Bioconductor with likely very little work required for
the Bioconductor Admins (ideally 100% of work would be done for
them).

Just a thought.

-Robert

Robert M Flight, PhD
Bioinformatics Research Associate
Puller of Rabbits from Hats
Research Parasite
Resource Center for Stable Isotope Resolved Metabolomics
Manager, Systems Biology and Omics Integration Journal Club
Markey Cancer Center
CC434 Roach Building
University of Kentucky
Lexington, KY

Twitter: @rmflight
Web: rmflight.github.io
ORCID: http://orcid.org/0000-0001-8141-7788
EM rflight79 at gmail.com
PH 502-509-1827

To call in the statistician after the experiment is done may be no
more than asking him to perform a post-mortem examination: he may
be able to say what the experiment died of. - Ronald Fisher

On Mon, Nov 20, 2017 at 8:51 AM Martin Morgan <martin.morgan at roswel
lpark.org> wrote:

On 11/19/2017 06:11 PM, Neumann, Steffen wrote:

Hi,

thanks for the answer, unfortunately
not exactly what I meant. To be more precise:

I want to point Galaxy to download (test) MS data files
from either the faahKO package, or msdata or mtbls2 in BioC.

For that I need an ftp/http/https link to the files,
and I was able to use

https://hedgehog.fhcrc.org/bioc-data/trunk/experiment/pkgs/msda

ta/inst/cdf/ko15.CDF

(which still works, but is probably outdated)
So is there anything equivalent in the new git setup ?

no, not from git.bioconductor.org.

Martin

Yours,
Steffen


On Fri, 2017-11-17 at 04:31 -0500, Martin Morgan wrote:

On 11/17/2017 04:08 AM, Neumann, Steffen wrote:

Hi,

I would like to have a URL to individual files
we have in BioC packages. This is useful
e.g. if I need test data from, say, the msdata package,
in another context.

In the SVN days, I was able to point directly
into the SVN repo with readonly:readonly access.
We also had the (now deprecated)?way to link to:
https://raw.githubusercontent.com/Bioconductor-mirror/xcms/ma

ster/D

ESCRIPTION

-> Is there anything comparable today ?

I think the git archive command might do the trick, but it

requires

ssh
(i.e., write) access to the repository

git archive --remote=git at git.bioconductor.org:packages/xcms \
? ??????master DESCRIPTION | tar -xf -

It could be as simple as a directory tree
of a working copies of all https://git.bioconductor.org/
served by the apache (or whatever) web server,
or (a bit cooler) a gitweb or similar http interface.

Wouldn't that be a fantastic christmas present
for us users and developers :-) ?

Yours,
Steffen

This email message may contain legally privileged and/or

confidential

information.??If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message

to the

intended recipient(s), you are hereby notified that any

disclosure,

copying, distribution, or use of this email message is
prohibited.??If you have received this message in error,

please

notify the sender immediately by e-mail and delete this email

message

from your computer. Thank you.

This email message may contain legally privileged
and/or...{{dropped:2}}