[Bioc-devel] Bioconductor Git/GitHub Mirrors

----- Original Message -----

From: "Leonardo Collado Torres" <lcollado at jhu.edu>
To: "Ryan C. Thompson" <rct at thompsonclan.org>
Cc: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <bioc-devel at r-project.org>
Sent: Tuesday, June 16, 2015 5:55:05 PM
Subject: Re: [Bioc-devel] Bioconductor Git/GitHub Mirrors

Hi,


This sounds great! Not having to setup all those bridges everytime a
new BioC version is released should be a great feature, plus keeping
the commit history tidy. And we can still use Travis or other things
like Slack notifications on our own bridges. I guess that it'd be
good
to remind users to pull new content from the bioc remote before
trying
to commit to svn.

Anyhow, great work!




I'm having a few problems with this new setup which hopefully are
easy
to iron out. Basically, I am stuck at the "git svn rebase && git svn
dcommit --add-author-from" step. See 2b) below.


1) My git was pointing to an older version even after updated with
brew. This was because I still had MacPorts around.
How to diagnose:
$ which git
Wrong version was pointing to /opt/local/bin/git instead of
/usr/local/bin/git Can also see by running
$ git --version ## if it doesn't match the version number you
installed from brew
Solution:
* Uninstall all of MacPorts following instructions from
https://guide.macports.org/chunked/installing.macports.uninstalling.html
* Follow part of
http://stackoverflow.com/questions/20648235/error-linking-git-to-homebrew
$ sudo brew prune
$ brew update
$ brew install git
$ brew unlink git && brew link git

Good to know. Things seem to work on the Macs I have tested with the vanilla git-svn that ships with the OS.

You might want to update subversion.
http://blog.victorquinn.com/fix-git-svn-in-mountain-lion might be
relevant for you.

I haven't needed to update svn.

2) Test case: derfinderHelper.

Smallest and simplest package I have, which won't matter much if
things break because it's stable.

a) You might want to delete update_remotes.sh after using it.
* I deleted the git-svn bridges to both
https://github.com/leekgroup/derfinderHelper and
https://github.com/leekgroup/derfinderHelper-release
* On https://github.com/leekgroup/derfinderHelper I removed bioc as a
collaborator and deleted the push-hook since it seems like we don't
need this anymore

Yes, you can delete the push hook. Leaving it does no harm as long as the bridge has been deleted (nobody will listen to the hook on the other end once the bridge is deleted).

* Download and ran update_remotes.sh as instructed. It doesn't seem
like we need it anymore after that point. If you leave it there, it
seems to give problems.

Can you explain what problems?

Easy to solve, but might want to clarify in the instructions.

I do this in a separate directory:

git clone https://github.com/Bioconductor/mirror.git

and then I have a repos containing the script where I can run 'git pull' if it ever changes. You could also symlink to it in /usr/local/bin. We'll see about changing the docs for this.

b) Following the section "Use Git Locally And Publicly" I get stuck
on
the "git svn rebase && git svn dcommit --add-author-from" step as
shown at https://gist.github.com/lcolladotor/ac8838b25955caeebcf1.
The
"CONFLICT (add/add): Merge conflict" message lead me to
http://stackoverflow.com/questions/19475387/how-to-handle-fix-git-add-add-conflicts
from where I could resolve the initial problem that I saw using "git
diff". But was basically the changes between the very first version
of
README.md and the very latest version of it.
There must be a way to not have to go through every single commit
and/or file one by one. Using "git rebase --continue" doesn't help.

I didn't have this problem. Here is what I did, which should be the same as what you did:

forked derfinderHelper to my personal (dtenenba) account

cloned the fork
ran the update_remotes.sh script in the fork

git checkout devel
git svn rebase
git merge master --no-edit
git svn rebase
git svn dcommit --add-author-from

None of these lines gave any errors and the last line said:

Committing to https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper ...

Doesn't seem like it actually needed to commit anything.

c) Given the issue with b, the devel branch already has a merge
commit. Using
http://stackoverflow.com/questions/927358/how-to-undo-the-last-commit
I cleaned things up. I don't think it's needed, but I don't want to
run into more issues.
$ git rebase --abort
$ git reset HEAD~1


As you can tell, I'm no "git rebase" expert. From
http://git-scm.com/docs/git-rebase it seems like the "ours" option
might be useful. But right now I prefer to get some pointers on how
to
proceed before playing around further :P

Well, since I can't reproduce this situation I'm not exactly sure what to tell you. If you know you want all the changes from git and none from svn (or vice versa) then the --ours and --theirs options are appropriate. But you might just want to start over and see if the problem goes away.

Here are my relevant versions:

$ git --version
git version 1.9.5 (Apple Git-50.3)
$ git svn --version
git-svn version 1.9.5 (Apple Git-50.3) (svn 1.7.19)
$ svn --version
svn, version 1.8.11 (r1643975)
$ uname -a
Darwin viper 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64

It does look like on this particular machine I have a non-standard svn:

$ which svn
/opt/subversion/bin/svn


Looks like it came from: https://www.wandisco.com/subversion/os/downloads

Thanks,
Leo


PS At the "Unable to determine upstream SVN information" section, I
believe that there is a typo. "git svn dcomit" should be "git svn
dcommit"

Will fix.

Thanks for the comments and for trying it out.

Dan

On Tue, Jun 16, 2015 at 3:20 PM, Ryan C. Thompson
<rct at thompsonclan.org> wrote:

This is great to hear. I sometimes want to delve into the source
code of a
package's internals, but doing so through the SVN web interface is
clunky.
Being able to use Github's repo browsing functionality for Bioc
packages is
great.


On 06/16/2015 12:00 PM, Dan Tenenbaum wrote:

Dear Bioconductors,

We're pleased to announce the availability of Bioconductor Git
Mirrors.
These are read-only GitHub repositories (available under
https://github.com/Bioconductor-mirror)
for every Bioconductor software package. These repositories are
synchronized with our
Subversion repository. Package maintainers (or anyone else) can
fork these
repositories
and do their development on the fork. Complete documentation of
the
mirrors is at

http://bioconductor.org/developers/how-to/git-mirror/

These mirrors supersede the Git-SVN bridge, which is now
deprecated.
Creation of new bridges is disabled and maintainers who are using
the
bridge
should migrate to the Git mirrors as soon as it's convenient, as
the
bridge
will eventually go away. Instructions for migrating can be found
at the
above link.

Some features of the new Git Mirrors, and why we feel they are a
better
solution than the Git-SVN bridge:

  - The mirrors contain complete commit history.
  - The mirrors contain release branches for Bioconductor 3.0 and
  3.1, and
    new releases will be added as they happen. You will no longer
    need
    separate repositories for release and devel.
  - Setup is easy and you no longer have to grant any permissions
  on your
    repository to other users. You can commit directly to
    Subversion
    using git-svn (https://git-scm.com/docs/git-svn).
  - Each git commit appears in the SVN log as a distinct SVN
  commit;
commits
    are no longer grouped together as they were with the bridge.
    Each git
    commit can be mapped to a specific svn commit, and vice versa.
  - The Git mirrors are much more reliable.
  - Use is flexible. You can use git locally (without GitHub) or
  you
    can use GitHub as well, to take advantage of all its social
    coding
features.
  - Using GitHub's code search, you can search the entire
  Bioconductor
codebase.
    Here's a sample search: https://goo.gl/jI92Ys
  - Subversion is fully supported and remains the cannonical
  repository;
use
    of Git and GitHub is optional.

We are excited about these new mirrors (brought to you by the hard
work of Jim Hester) and we hope you are too. Questions and
comments are
welcome on the bioc-devel mailing list.

Dan