Skip to content

[Bioc-devel] Confusion with how to maintain release/devel files on local computer.

15 messages · Gabriel Becker, Laurent Gatto, Stephanie M. Gogarten +3 more

#
I?m confused with development process.

At first, I need to have a folder with accepted packaged. Then I need to pull
origion RELEASE_3_6?

Then in another folder, I need to pull origion master?

So that by opening each folder, I know what I?m editing.
Also during push, I need to be careful about where I?m pushing changes.
Origion is bioc?s git address of my package whereas master is the package directory in GitHub?

Am I getting it correct?
Is there anywhere that contains whole the process and codes in steps?

Best regards,
Arman
#
Arman,

Not on the Bioc team per se, but I would say only have a checkout of the
release branch when you need it, ie a bug is reported, you have fixed it in
devel, and you are ready to push the very narrow bugfix to release. I only
keep "master" checkouts of my packages on a permanent basis.

You generally shouldn't need a checkout of release, imho, because no
development should be happening there with exception of the case above.

Hope that helps,
~G

On Wed, Nov 1, 2017 at 1:36 PM, Arman Shahrisa <shahrisa.arman at hotmail.com>
wrote:

  
    
#
On 11/01/2017 04:42 PM, Gabe Becker wrote:
In case Arman is not aware, the basic scenarios are at

   https://bioconductor.org/developers/how-to/git

the most straight-forward of which (a clone of the Bioconductor git 
repository, no GitHub mirror) is

   https://bioconductor.org/developers/how-to/git/maintain-bioc-only/

Generally, one has only a single local git repository, switching between 
branches within that repository.

Martin
This email message may contain legally privileged and/or...{{dropped:2}}
#
On 1 November 2017 20:36, Arman Shahrisa wrote:

            
No, it all happens in the same folder, but switching between branches
using git. Here's an example of one of my own packages. The first
command list all available branches (all, using -a, means also
remote-only branches). My current branch is noted with an *, and I also
have a feature branch called writeMSData, which also lives on GitHub
(https://github.com/lgatto/MSnbase/, but that's optional).

$ git branch -a
  devel
* master
  writeMSData
  remotes/origin/HEAD -> origin/master
  remotes/origin/centroiding
  remotes/origin/fixBracketSubset
  remotes/origin/issue82
  remotes/origin/master
  remotes/origin/orbifilter
  remotes/origin/processingData
  remotes/origin/removePrecMz
  remotes/origin/writeMSData
  remotes/upstream/RELEASE_2_10
  remotes/upstream/RELEASE_2_11
  remotes/upstream/RELEASE_2_12
  remotes/upstream/RELEASE_2_13
  remotes/upstream/RELEASE_2_14
  remotes/upstream/RELEASE_2_8
  remotes/upstream/RELEASE_2_9
  remotes/upstream/RELEASE_3_0
  remotes/upstream/RELEASE_3_1
  remotes/upstream/RELEASE_3_2
  remotes/upstream/RELEASE_3_3
  remotes/upstream/RELEASE_3_4
  remotes/upstream/RELEASE_3_5
  remotes/upstream/master

As you can see (and as specified by Gabe in his earlier reply), I
haven't have pulled all Bioconductor releases. master points to GitHub's
origin/master branch, and devel points to Bioconductor's
upstream/master. As you can see above, I haven't got the latest release
references yet. I can do this with

$ git fetch --all
Fetching origin
Fetching upstream
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 4), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
* [new branch]      RELEASE_3_6 -> upstream/RELEASE_3_6
   b680678..a98138c  master     -> upstream/master

And now

$ git branch -a  
  devel
* master
  writeMSData
  remotes/origin/HEAD -> origin/master
  remotes/origin/centroiding
  remotes/origin/fixBracketSubset
  remotes/origin/issue82
  remotes/origin/master
  remotes/origin/orbifilter
  remotes/origin/processingData
  remotes/origin/removePrecMz
  remotes/origin/writeMSData
  remotes/upstream/RELEASE_2_10
  remotes/upstream/RELEASE_2_11
  remotes/upstream/RELEASE_2_12
  remotes/upstream/RELEASE_2_13
  remotes/upstream/RELEASE_2_14
  remotes/upstream/RELEASE_2_8
  remotes/upstream/RELEASE_2_9
  remotes/upstream/RELEASE_3_0
  remotes/upstream/RELEASE_3_1
  remotes/upstream/RELEASE_3_2
  remotes/upstream/RELEASE_3_3
  remotes/upstream/RELEASE_3_4
  remotes/upstream/RELEASE_3_5
  remotes/upstream/RELEASE_3_6
  remotes/upstream/master

If I want to modify the development branch (i.e. Bioconductor's
upstreams/master), then I checkout devel (that's how I named it
locally), do changes and push.

$ git checkout devel
## do stuff
$ git push

Same principle for other branches.
No - I suggest you read a bit about git (GitHub is a web interface using
git) to familiarise yourself with the concepts and vocabulary.
All the setup and more details are provided in 

  https://github.com/bioconductor/bioc_git_transition/

in particular the FAQ and all the scenarios at the bottom

  https://github.com/Bioconductor/bioc_git_transition/blob/master/doc/faq.md

Best wishes,

Laurent

  
    
#
One possible point of confusion: Laurent's workflow includes maintaining 
separate branches "master" and "devel", which he syncs to his own Github 
repo and Bioconductor's git repo respectively. However, the 
documentation on the bioc website 
(https://bioconductor.org/developers/how-to/git/) assumes that you have 
only one "master" branch that you push to both remotes.
On 11/1/17 2:33 PM, Laurent Gatto wrote:
#
I cloned the package into a clean directory. Then I followed ?New package workflow? strategy. By checking Description file while switching between branches, I can easily see the version change which is correct. If I understand it correctly, GitHub can only contain changes I make to master branch. If I want to push changes to GitHub as well, what should I do?

Best regards,
Arman


From: Stephanie M. Gogarten<mailto:sdmorris at uw.edu>
Sent: Thursday, November 2, 2017 02:19
To: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>
Cc: bioc-devel<mailto:bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.

One possible point of confusion: Laurent's workflow includes maintaining
separate branches "master" and "devel", which he syncs to his own Github
repo and Bioconductor's git repo respectively. However, the
documentation on the bioc website
(https://bioconductor.org/developers/how-to/git/) assumes that you have
only one "master" branch that you push to both remotes.
On 11/1/17 2:33 PM, Laurent Gatto wrote:

  
  
#
I found how I can create another remote with GitHub using Rstudio. How can I merge master and GitHub?

Best regards,
Arman

From: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>
Sent: Thursday, November 2, 2017 03:23
To: Stephanie M. Gogarten<mailto:sdmorris at uw.edu>; bioc-devel<mailto:bioc-devel at r-project.org>
Subject: RE: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.

I cloned the package into a clean directory. Then I followed ?New package workflow? strategy. By checking Description file while switching between branches, I can easily see the version change which is correct. If I understand it correctly, GitHub can only contain changes I make to master branch. If I want to push changes to GitHub as well, what should I do?

Best regards,
Arman


From: Stephanie M. Gogarten<mailto:sdmorris at uw.edu>
Sent: Thursday, November 2, 2017 02:19
To: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>
Cc: bioc-devel<mailto:bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.

One possible point of confusion: Laurent's workflow includes maintaining
separate branches "master" and "devel", which he syncs to his own Github
repo and Bioconductor's git repo respectively. However, the
documentation on the bioc website
(https://bioconductor.org/developers/how-to/git/) assumes that you have
only one "master" branch that you push to both remotes.
On 11/1/17 2:33 PM, Laurent Gatto wrote:

  
  
#
As I plan to "fix a bug" in release version of BiocSklearn, I tried

%vjcair> git checkout remotes/upstream/RELEASE_3_6

Here is the result; what would I do to make some changes, alter DESCRIPTION
version to 1.0.1, commit and push so that the build system

has 1.0.1?

Note: checking out 'remotes/upstream/RELEASE_3_6'.


You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by performing another checkout.


If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -b with the checkout command again. Example:


  git checkout -b <new-branch-name>


HEAD is now at 98fabcd... bump x.y.z versions to even y prior to creation
of RELEASE_3_6 branch

On Wed, Nov 1, 2017 at 8:19 PM, Arman Shahrisa <shahrisa.arman at hotmail.com>
wrote:

  
  
#
On 11/01/2017 08:54 PM, Vincent Carey wrote:
Start over by getting back to master

   git checkout master

then check out the release branch

   git checkout RELEASE_3_6
   git branch # verify -- RELEASE_3_6
   git log    # verify -- should see history through recent branch

edit DESCRIPTION, etc, and make a local commit

   git add DESCRIPTION # etc -- add material to be committed
   git commit          # make the local commit, enter commit msg, etc

What your next commands are depends on how you've named the remotes. For 
instance, I happened to call the remote associated with 
git.bioconductor.org 'origin'

$ git remote -v
origin	git at git.bioconductor.org:packages/BiocSklearn (fetch)
origin	git at git.bioconductor.org:packages/BiocSklearn (push)

I guess you named the remote 'upstream'; maybe you also have a remote 
'origin' associated with github.

I'll push my committed changes to bioconductor

   git push origin RELEASE_3_6

change 'origin' to push to a different remote, as appropriate.

Martin
This email message may contain legally privileged and/or...{{dropped:2}}
#
On 11/01/2017 07:53 PM, Arman Shahrisa wrote:
It helps to be more precise here. Were did you clone the package from? 
In the simplest case it would be

   git clone git at git.bioconductor.org:packages/your-package

and you'd have a single remote named 'origin'

   git remote -v

You can add or rename remotes, e.g.,

   git remote rename origin upstream
   git remote add origin git at github.com:your-name/your-package

and you can pull (or fetch and then merge) from each

   git pull upstream  # pull changes from 'upstream' remote
   git pull origin    # pull changes from 'origin'

Study the how-to documents for different scenarios, and be precise as 
possible with what you are trying to do. Also, when learning, it can be 
comforting to know that removing your local repository from your disk is 
not the end of the world, you can just start over with a new clone from 
git.bioconductor.org or from github.
You can do whatever you want with your local clone, and with github. The 
Bioconductor git repository allows you to push only to 'master' and the 
current release, RELEASE_3_6. Suppose you'd made changes to master, and 
you named the remotes 'origin' (for github) and 'upstream' (for 
bioconductor). Then after after adding (git add ...) and committing (git 
commit ...) to the local repository, you'd push to bioconductor with

   git push upstream master

and to github with

   git push origin master

Martin
This email message may contain legally privileged and/or...{{dropped:2}}
#
I can see two branches master and releas_3_6. I created GMaster to
connect it with GitHub. But as far as it looks like, it?s only a local directory.
Also, I don?t know weather I can remove it or not. Should upsteam appear here?

Best regards,
Arman

From: Martin Morgan<mailto:martin.morgan at roswellpark.org>
Sent: Thursday, November 2, 2017 05:00
To: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>; Stephanie M. Gogarten<mailto:sdmorris at uw.edu>; bioc-devel<mailto:bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.
On 11/01/2017 07:53 PM, Arman Shahrisa wrote:
It helps to be more precise here. Were did you clone the package from?
In the simplest case it would be

   git clone git at git.bioconductor.org:packages/your-package

and you'd have a single remote named 'origin'

   git remote -v

You can add or rename remotes, e.g.,

   git remote rename origin upstream
   git remote add origin git at github.com:your-name/your-package

and you can pull (or fetch and then merge) from each

   git pull upstream  # pull changes from 'upstream' remote
   git pull origin    # pull changes from 'origin'

Study the how-to documents for different scenarios, and be precise as
possible with what you are trying to do. Also, when learning, it can be
comforting to know that removing your local repository from your disk is
not the end of the world, you can just start over with a new clone from
git.bioconductor.org or from github.
You can do whatever you want with your local clone, and with github. The
Bioconductor git repository allows you to push only to 'master' and the
current release, RELEASE_3_6. Suppose you'd made changes to master, and
you named the remotes 'origin' (for github) and 'upstream' (for
bioconductor). Then after after adding (git add ...) and committing (git
commit ...) to the local repository, you'd push to bioconductor with

   git push upstream master

and to github with

   git push origin master

Martin
This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2017-11-02_17-35-42.png
Type: image/png
Size: 10716 bytes
Desc: 2017-11-02_17-35-42.png
URL: <https://stat.ethz.ch/pipermail/bioc-devel/attachments/20171102/4e75211a/attachment.png>
#
On 11/02/2017 10:11 AM, Arman Shahrisa wrote:
Again, you need to be very precise here.

There is no branch releas_3_6.

I don't know what 'it' is -- I guess your local repository that you 
cloned, but where did you clone it from?

I don't know what you mean by 'created GMaster'
the cloned repository is a local directory, yes.
If you remove it, then any changes not 'pushed' to an online repository 
will be lost.

'upstream' would appear with

   git remote -v

if you had added an 'upstream' remote

   git remote add upstream git at git.bioconductor.org:packages/your-package

or if you had renamed an existing remote

   git rename origin upstream

'origin' and 'upstream' are just names; you can call remotes whatever 
you want.

It would be helpful to report as carefully as possible what your 
configure and commands are. For instance instead of your message, you 
could say

I can see two branches

$ git branch  -a
   RELEASE_3_6
* master

I have created a remote connecting to GitHub

$ git remote -v
origin	git at github.com:Bioconductor/your-package (fetch)
origin	git at github.com:Bioconductor/your-package (push)
upstream	git at git.bioconductor.org:packages/your-package (fetch)
upstream	git at git.bioconductor.org:packages/your-package (push)

and so on.

Martin
This email message may contain legally privileged and/or...{{dropped:2}}
#
I repeated all the process again.
I opened git bash as an administrator and ran commands described here:

https://bioconductor.org/developers/how-to/git/maintain-bioc-only/
The I ran this one:
Then these commands from https://bioconductor.org/developers/how-to/git/pull-upstream-changes/
I have attached an screenshot of what I have now. The following is the
Command-line output:
Thank you very much for your help.

Best regards,
Arman

From: Martin Morgan<mailto:martin.morgan at roswellpark.org>
Sent: Thursday, November 2, 2017 17:58
To: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>; bioc-devel<mailto:bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.
On 11/02/2017 10:11 AM, Arman Shahrisa wrote:
Again, you need to be very precise here.

There is no branch releas_3_6.

I don't know what 'it' is -- I guess your local repository that you
cloned, but where did you clone it from?

I don't know what you mean by 'created GMaster'
the cloned repository is a local directory, yes.
If you remove it, then any changes not 'pushed' to an online repository
will be lost.

'upstream' would appear with

   git remote -v

if you had added an 'upstream' remote

   git remote add upstream git at git.bioconductor.org:packages/your-package

or if you had renamed an existing remote

   git rename origin upstream

'origin' and 'upstream' are just names; you can call remotes whatever
you want.

It would be helpful to report as carefully as possible what your
configure and commands are. For instance instead of your message, you
could say

I can see two branches

$ git branch  -a
   RELEASE_3_6
* master

I have created a remote connecting to GitHub

$ git remote -v
origin  git at github.com:Bioconductor/your-package (fetch)
origin  git at github.com:Bioconductor/your-package (push)
upstream        git at git.bioconductor.org:packages/your-package (fetch)
upstream        git at git.bioconductor.org:packages/your-package (push)

and so on.

Martin
This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2017-11-02_18-40-51.png
Type: image/png
Size: 7009 bytes
Desc: 2017-11-02_18-40-51.png
URL: <https://stat.ethz.ch/pipermail/bioc-devel/attachments/20171102/a901847d/attachment.png>
#
Thank you for the detailed description, it is much easier to understand 
what is going on. I have made some comments below
On 11/02/2017 11:14 AM, Arman Shahrisa wrote:
it is not necessary or recommended to run these commands as an 
administrator; generally all activities should be done as a regular user.

Likely you need, as a regular user, to regenerate your ssh key pair, and 
to re-submit the public key to Bioconductor. Alternatively, perhaps as a 
regular user you already have an ssh key pair, and you can submit the 
public key part of the key pair. Remember that this will take at least 
24 hours to process.
This workflow means that you want to maintain only the Bioconductor 
version of your repository, you do not want to maintain the github 
version of your repository that you used during package submission.
ok, all of the above looks good. You cloned the repository, and your ssh 
key was recognized.
This step is not required. It is not part of the workflow you cited. 
Earlier, when you issued the command 'git remote -v' git indicated that 
you already had a remote named 'origin' and referencing the git server 
git at git.bioconductor.org. What the command above does is to add a second 
remote pointing to git.bioconductor.org, but this one named 'upstream'.

I can see that the documentation in this work flow needs to be adjusted, 
because in other

It would be appropriate to rename the remote, e.g.,

   git remote rename origin upstream

nonetheless, issuing the command should not cause a problem.

It seems like at this stage you should continue with the workflow that 
you started, i.e.,

   - 'Commit changes to your local repository' to develop a new feature 
or fix a bug, including checking that your code changes are correct by 
running R CMD build and R CMD check using the 'devel' version of 
Bioconductor, and

   - 'Push your local changes to the Bioconductor repository' to make 
your code changes available to the Bioconductor build server. Remember 
that builds run once a day, and that you need to confirm that the build 
has been successful by visiting http://bioconductor.org/checkResults/.

The steps below are not necessary for you at the moment.

Martin
This email message may contain legally privileged and/or...{{dropped:2}}
#
Thank you very much for all your assistance. Unfortunately, git bash must be run as administrator,
otherwise user won?t be able to change the pwd from partition C to another partition (It will remain in
the previous directory on partition C). This is why I had to run it as administrator.


Best regards,
Arman

From: Martin Morgan<mailto:martin.morgan at roswellpark.org>
Sent: Friday, November 3, 2017 13:23
To: Arman Shahrisa<mailto:shahrisa.arman at hotmail.com>; bioc-devel<mailto:bioc-devel at r-project.org>
Subject: Re: [Bioc-devel] Confusion with how to maintain release/devel files on local computer.

Thank you for the detailed description, it is much easier to understand
what is going on. I have made some comments below
On 11/02/2017 11:14 AM, Arman Shahrisa wrote:
it is not necessary or recommended to run these commands as an
administrator; generally all activities should be done as a regular user.

Likely you need, as a regular user, to regenerate your ssh key pair, and
to re-submit the public key to Bioconductor. Alternatively, perhaps as a
regular user you already have an ssh key pair, and you can submit the
public key part of the key pair. Remember that this will take at least
24 hours to process.
This workflow means that you want to maintain only the Bioconductor
version of your repository, you do not want to maintain the github
version of your repository that you used during package submission.
ok, all of the above looks good. You cloned the repository, and your ssh
key was recognized.
This step is not required. It is not part of the workflow you cited.
Earlier, when you issued the command 'git remote -v' git indicated that
you already had a remote named 'origin' and referencing the git server
git at git.bioconductor.org. What the command above does is to add a second
remote pointing to git.bioconductor.org, but this one named 'upstream'.

I can see that the documentation in this work flow needs to be adjusted,
because in other

It would be appropriate to rename the remote, e.g.,

   git remote rename origin upstream

nonetheless, issuing the command should not cause a problem.

It seems like at this stage you should continue with the workflow that
you started, i.e.,

   - 'Commit changes to your local repository' to develop a new feature
or fix a bug, including checking that your code changes are correct by
running R CMD build and R CMD check using the 'devel' version of
Bioconductor, and

   - 'Push your local changes to the Bioconductor repository' to make
your code changes available to the Bioconductor build server. Remember
that builds run once a day, and that you need to confirm that the build
has been successful by visiting http://bioconductor.org/checkResults/.

The steps below are not necessary for you at the moment.

Martin
This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.