[Bioc-devel] Workflows are now in git (and other Important workflow-related changes)
Thank you Andrzej for the info! ## Re-point (1) Regarding my (1), I got my github repo in sync following http://bioconductor.org/developers/how-to/git/sync-existing-repositories/ and http://bioconductor.org/developers/how-to/git/abandon-changes/#force-bioconductor--to-github- It all worked without problems. Thanks again for the great docs! ## on.bioc auto-detection At https://github.com/LieberInstitute/recountWorkflow/commit/b456fbbe6ddbeda297ef03b79de9e004f28c0b97#diff-a87d0071c97357f224ad6608f837fbbfR45 I'm using on.bioc <- knitr::opts_knit$get("rmarkdown.pandoc.to") != 'latex' to detect automatically whether I'm rendering the output for Bioc or not. For F1000 I'll use BiocWorkflowTools::f1000_article as the output by calling rmarkdown::render() manually. I thought this little code would help others who want to have a single repo for both BioC and F1000. ## Re-point (5)
From Lori's email that she sent today, I can see at
http://bioconductor.org/checkResults/3.7/workflows-LATEST/recountWorkflow/malbec2-buildwebvig.html how the final Bioc web version is being built. That is: /home/biocbuild/bbs-3.7-bioc/R/bin/R -q -e 'rmarkdown::render(".buildwebvig/recountWorkflow/recount-workflow.Rmd", output_format="BiocStyle:::html_fragment")' && /home/biocbuild/bbs-3.7-bioc/R/bin/R -q -e 'knitr::purl(".buildwebvig/recountWorkflow/recount-workflow.Rmd", ".buildwebvig/recountWorkflow/recount-workflow.R")' I can see the R version being used at http://bioconductor.org/checkResults/3.7/workflows-LATEST/index.html although not the bioc version (that I can infer from the package loading messages, like clusterProfiler's one). This only matters in those 6 months when release/devel are being built on the same R version. Now that I have the commands and versions, if I ever see anything weird on the rendered Bioc web version I have a way to test things myself instead of relying on the bioc builds. Since this is a bit different than the rest of Bioc, maybe it'd be good to document the build commands somewhere else. Again, let me thank all of you for your work! I'm really excited about the new workflow build chain and how it'll simplify maintaining/writing workflows! Best, Leo
On Tue, Apr 3, 2018 at 6:38 AM, Andrzej Ole? <andrzej.oles at gmail.com> wrote:
Dear Leo, thanks for you inquiry. Re points 2-4: there is no need to use the token file .html_output anymore. This was an intermediate solution meant for the transition period to new output format. Now all workflow vignettes for the the website are build using this new format, regardless the `output` setting in the vignette YAML header. Note that the format set in the vignette is still used to render the document distributed with the package tarball. Cheers, Andrzej On Mon, Apr 2, 2018 at 4:32 PM, Leonardo Collado Torres <lcollado at jhu.edu> wrote:
Hi, Thank you for all the work in this transition Andrzej, Lori, Valerie, Nitesh and Herv?! I have a few questions about the new system. 1) I already had a github repo (that I manually kept in sync with the subversion one) for the recountWorkflow. Should I just follow the instructions at http://bioconductor.org/developers/how-to/git/sync-existing-repositories/? I guess that I'll need to do this too http://bioconductor.org/developers/how-to/git/abandon-changes/#force-bioconductor--to-github- 2) Is repo/vignettes/.html_output still needed? 3) Related to (2), the .html_output file used to ignore the `output` line in the vignette YAML. I got an email from Lori saying that output: BiocWorkflowTools::f1000_article is not working for recountWorfkflow for now (no html vignette). My guess is that the `output` line now matters, and in my case, that would build a pdf file which is ignored due to https://github.com/LieberInstitute/recountWorkflow/blob/master/.Rbuildignore#L12 in my .Rbuildignore. 4) Related to the last 2. What setup do you recommend so I can maintain a single git repo that produces the files needed for uploading them to F1000research? Before this transition, I had to maintain 3 repos: the subversion one for Bioc, my git copy (well, main), and another one with the files setup for F1000. At some point in the past, Andrzej recommended that I use "output: BiocWorkflowTools::f1000_article" to try to have all the files in 1 place. I imagine that I would still need some ifelse() statements for sentences like the one I have at https://github.com/LieberInstitute/recountWorkflow/blob/master/vignettes/recount-workflow.Rmd#session-information I think that this question could be generalized and documented in the Bioc website. 5) Is there anything else I should be aware of so I can reproduce any errors? In the past, the Bioc workflow system would trigger a build immediately after a new push, and due to the custom way workflows were built (like .html_output), I relied on these builds to check if my changes had worked or not: specially early on where I had issues with figures, links to figures and figure captions. Best, Leo On Mon, Apr 2, 2018 at 7:42 AM, Shepherd, Lori <Lori.Shepherd at roswellpark.org> wrote:
For now the page at https://www.bioconductor.org/help/workflows/ will remain and be updated accordingly however there is discussion about having this page be removed or redirected. In the next few days I hope to have the new landing pages up and running and will make an announcement when they are live. The new changes involve having a link to the workflows on the biocViews page, that will directly link to a workflow package landing page (like the software, annotation, and experiment package do). Until the release your changes will be visible in devel which will transition over to the new release just as done for software packages - The workflow are built monday, wednesday, and friday and the new landing pages will reflect when the package is propagated like the other types of packages. http://bioconductor.org/checkResults/3.7/workflows-LATEST/ Lori Shepherd Bioconductor Core Team Roswell Park Cancer Institute Department of Biostatistics & Bioinformatics Elm & Carlton Streets Buffalo, New York 14263
________________________________ From: Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of Aaron Lun <alun at wehi.edu.au> Sent: Sunday, April 1, 2018 2:17:09 PM To: Herv? Pag?s; bioc-devel at r-project.org Subject: Re: [Bioc-devel] Workflows are now in git (and other Important workflow-related changes) Thanks everybody, this is much appreciated. On that note, will the compiled workflows shown at: https://www.bioconductor.org/help/workflows/ ... be updated to reflect changes in the git repositories for the workflows? Or will the workflow page just directly link to the landing page for each package? This would be very convenient, not least because it will avoid me having to make pull requests to the bioconductor.org repository every time I want to change my workflow blurb. I'm in the process of fixing and updating my various workflows, and I'm wondering when/how my changes will be visible to users. Cheers, Aaron On Fri, 2018-03-30 at 13:10 -0700, Herv? Pag?s wrote: To the authors/maintainers of the workflows: Following the svn-to-git migration of the software and data experiment packages last summer, we've completed the migration of the workflow packages. The canonical location for the workflow source code now is git.bioconductor.org Please use your git client to access/maintain your workflow the same way you would do it for a software or data-experiment package. We've also migrated the workflows to our in-house build system. Starting with Bioc 3.7, the build report for the devel versions of the workflows can be found here: https://bioconductor.org/checkResults/devel/workflows-LATEST/ We run these builds every other day (Mondays, Wednesdays, Fridays). Because of limited build resources, we now run the data-experiment builds on Sundays, Tuesdays, and Thursdays only (instead of daily). The links to the package landing pages are not working yet. This will be addressed in the next few days. Please address any error you see on the report for the workflow you maintain. Note that, from now on, we're also following the same version scheme for these packages as for the software and data-experiment packages. That is, we're using an even y (in x.y.z) in release and an odd y in devel. We'll take care of bumping y at release time (like we do for software and data-experiment packages). After the next Bioconductor release (scheduled for May 1), we'll start building the release versions of the workflows in addition to the devel versions. The build report for the release versions will be here: https://bioconductor.org/checkResults/release/workflows-LATEST/ Finally, please note that with the latest version of BiocInstaller (1.29.5), workflow packages can be installed with biocLite(), like any other Bioconductor package. We'll deprecate the old mechanism (workflowInstall()) at some point in the future. Thanks to Andrzej, Lori, Nitesh, and Valerie for working on this migration. Let us know if you have any question about this. H. _______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]] _______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
_______________________________________________ Bioc-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel