Hi Leo,
first of all, many thanks for your efforts in troubleshooting workflow
build issues and the detailed description of your findings. Also,
congratulations on the successful publication of your new workflow on the
Bioconductor website!
We are now beta testing a new workflow build engine which enables some
features not possible to achieve in the previous approach, such as
cross-references. The goal is to properly render the output for the BioC
website directly from the Rmd file authored for BiocWorkflowTools::f1000_article
without any special tweaks or hacks. This new engine is enabled by adding
the special '.html_output' token file in the vignettes/ dir. Ideally, you
should be able to get decent output from the original Rmd file which you
used for the F1000 Research submission without any manual Bioc-specific
modifications. Note that in principle you could even leave your primary
document output format as `BiocWorkflowTools::f1000_article`. This is
because the workflows for the website are build through a call to
`rmarkdown::render` by specifying a custom render format, so the output
format set in the document header is discarded anyway; it is important only
for the vignette included in the package tarball. But it's also fine to use
either of the formats provided by BiocStyle.
It shouldn't be necessary to use the captioner package anymore, as figure
numbering is now handled by bookdown. Gosia used it in cytofWorkflow before
I enabled the new html engine for her workflow, which I did only yesterday.
Re the specific builder issues: I've deleted the /tmp/udcCache dir on Mac,
but it didn't help. I'm not sure yet what could be the problem there. The
citeproc conversion error on Windows might be because of the older pandoc
version (1.17.2), will look into this. The convert warnings appear because
the new BiocStyle formats set the `knitr::opts_chunk(crop=TRUE)` to crop
out excessive white space around plots, will try to sanitize this a little
bit too.
Cheers,
Andrzej
On Thu, Aug 31, 2017 at 8:15 PM, Leonardo Collado Torres <lcollado at jhu.edu
Hi,
I recently got a workflow accepted and I've been trying to get the
workflow builder to successfully complete on Mac and Windows which is
based on bioc-release. Note that the workflow did build properly on
those operating systems using the SPB (bioc-devel).
## Linux: working
After looking at
https://hedgehog.fhcrc.org/bioconductor/trunk/madman/workflo
ws/cytofWorkflow
and https://hedgehog.fhcrc.org/bioconductor/trunk/madman/workflo
ws/rnaseqGene
I was able to:
(1) get the figure references and captions working using captioner
(2) actually get the figures to show
(3) use the new layout (thanks to vignettes/.html_output)
Since the build completes on Linux,
http://bioconductor.org/help/workflows/recountWorkflow/ is now live.
Only the links to the Mac and Windows binaries fail.
## Mac: could be a permissions issue
The issue with Mac is (details at
http://docbuilder.bioconductor.org:8080/job/recountWorkflow/
label=vagrantmac/9/console)
is that I eventually run into this error:
Quitting from lines 735-747 (recount-workflow.Rmd)
Error: processing vignette 'recount-workflow.Rmd' failed with diagnostics:
UCSC library operation failed
Execution halted
The referenced lines break because of:
regions <- expressed_regions("SRP045638", "chr21", cutoff = 5L,
maxClusterGap = 3000L)
This code uses rtracklayer::import.bw() with a URL that gets
forwarded. I made sure that I am requiring the latest bioc-release
rtracklayer in the description file, so this leads me to think that
this issue is a repeat of
https://stat.ethz.ch/pipermail/bioc-devel/2016-August/009599.html
where the solution involved changing some permissions. Dan stated in
that thread (3rd email): "Actually it looks like it was a permissions
issue with the directory /tmp/udcCache. I removed this directory (as
superuser) and that error no longer happens."
Does this sound like something that could be happening in the Mac builder?
## Windows: figure and pandoc-citeproc issues
In Windows, I see warnings like this:
Invalid Parameter - -trim
Warning: running command 'C:\Windows\system32\cmd.exe /c convert
"exondeanalysis1-1.png" -trim "exondeanalysis1-1.png"' had status 4
Warning in shell(paste(c(cmd, args), collapse = " ")) :
'convert "exondeanalysis1-1.png" -trim "exondeanalysis1-1.png"'
execution failed with error code 4
Details at http://docbuilder.bioconductor.org:8080/job/recountWorkflow/
label=winbuilder1/9/console.
I originally thought that it was related to the figure paths, which is
why I added this knitr code:
knitr::opts_chunk$set(fig.path = "")
But that didn't resolve the issue. I get these warnings only with
BiocStyle::html_document2 and not with BiocStyle::html_document
(http://docbuilder.bioconductor.org:8080/job/recountWorkflow
/label=winbuilder1/7/console
using v0.99.28).
Ultimately, the warnings might not matter. Though I don't see them on
the build reports for other workflows (cytofWorkflow, rnaseqGene).
The Windows builds fail (with BiocStyle::html_document2 or
BiocStyle::html_document) with error messages like this:
pandoc.exe: Error running filter pandoc-citeproc Filter returned error
status 1073807366 Warning: running command
'"C:/Progra~2/Pandoc/pandoc" +RTS -K512m -RTS recount-workflow.utf8.md
--to html --from
markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash
--output recount-workflow.html --smart --email-obfuscation none
--self-contained --standalone --section-divs --table-of-contents
--toc-depth 3 --template
"C:\Windows\TEMP\RtmpchKz3E/BiocStyle/template.html" --no-highlight
--variable highlightjs=1 --number-sections --css
"C:\PROGRA~1\R\R-34~1.0\library\BIOCST~1\RESOUR~1\html\BIOCON~2.CSS"
--variable "theme:bootstrap" --include-in-header
"C:\Windows\TEMP\RtmpchKz3E\rmarkdown-str3ec7c88cea.html" --mathjax
--variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?c
onfig=TeX-AMS-MML_HTMLorMML"
--bibliography recount-workflow.bib --filter pandoc-citeproc' had
status 83 Error: processing vignette 'recount-workflow.Rmd' failed
with diagnostics: pandoc document conversion failed with error 83
Execution halted
(from http://docbuilder.bioconductor.org:8080/job/recountWorkflow/
label=winbuilder1/9/console)
Googling the error pointed me towards having weird symbols on the .bib
file. So I removed some accents and the like, but I still get this
error. I am using a csl file for the bibliography and like I said,
this works with the SPB whose last build report was
http://bioconductor.org/spb_reports/recountWorkflow_buildrep
ort_20170807114418.html
(v0.99.18). There's not much different between that version and
0.99.30 as you can see at
https://github.com/LieberInstitute/recountWorkflow/commits/master **.
The Linux machine has pandoc 1.19.2.1 installed
(http://bioconductor.org/help/workflows/recountWorkflow/#ses
sion-information)
and I guess that the Windows one has the same version. But maybe that
could be the issue. BiocStyle 2.5.15 is installed in the Linux machine
too (Bioconductor/BiocStyle at 2a1ba75) which is newer than the latest
bioc-release (2.4.1), but again, I assume that the Windows machine has
the same version.
Anyhow, I haven't been able to fix these issues and was wondering if
anyone else had ideas that could resolve them. Regardless, I'm happy
that I got http://bioconductor.org/help/workflows/recountWorkflow/ up
and looking good ^^. It's just the links to the Mac/Windows
binaries...
Thank you,
Leo
** I've been pushing changes via svn, I know that workflows are not
part of the git transition. I just keep everything in sync manually
with the GitHub repo.
Leonardo Collado Torres, Ph. D., Data Scientist
Lieber Institute for Brain Development
855 N Wolfe St, Suite 300
Baltimore, MD 21205
Website: http://lcolladotor.github.io