Skip to content

[Bioc-devel] R-devel for Windows: Rtools3.5 vs Rtools4.0

10 messages · Martin Morgan, Gordon K Smyth, Hervé Pagès +1 more

Original
Gordon K Smyth
# 
Dear Biocore team,

How is Bioconductor handling the transition to the new R for Windows toolchain?

The R-Devel build for Windows currently available from CRAN uses Jerome Oom's new gcc 8.3.0 toolchain from rtools40, whereas R-Devel builds for Windows downloaded from CRAN a couple of months ago used the older rtool35 toolchain.

It appears that the Bioconductor developmental repository is using rtools35, which means that BiocManager::install is incompatible with the R-Devel build for Windows now on CRAN but works with R-Devel from a month or two ago.

When Bioconductor 3.11 is released, will it be the rtools40 toolchain or rtools35?

Thanks
Gordon

------------------------------------------
Professor Gordon K Smyth
Joint Head, Bioinformatics Division
Walter and Eliza Hall Institute of Medical Research

_______________________________________________

The information in this email is confidential and intend...{{dropped:12}}
Martin Morgan
# 
The Bioconductor 3.11 builds transitioned in the last week to using the new tool chain, following CRAN's lead.

As a sanity check (ours!) what makes you say
?

Martin Morgan

?On 4/19/20, 10:41 PM, "Bioc-devel on behalf of Gordon K Smyth" <bioc-devel-bounces at r-project.org on behalf of smyth at wehi.edu.au> wrote:

    Dear Biocore team,
    
    How is Bioconductor handling the transition to the new R for Windows toolchain?
    
    The R-Devel build for Windows currently available from CRAN uses Jerome Oom's new gcc 8.3.0 toolchain from rtools40, whereas R-Devel builds for Windows downloaded from CRAN a couple of months ago used the older rtool35 toolchain.
    
    It appears that the Bioconductor developmental repository is using rtools35, which means that BiocManager::install is incompatible with the R-Devel build for Windows now on CRAN but works with R-Devel from a month or two ago.
    
    When Bioconductor 3.11 is released, will it be the rtools40 toolchain or rtools35?
    
    Thanks
    Gordon
    
    ------------------------------------------
    Professor Gordon K Smyth
    Joint Head, Bioinformatics Division
    Walter and Eliza Hall Institute of Medical Research
    
    _______________________________________________
    
    The information in this email is confidential and intend...{{dropped:12}}
    
    _______________________________________________
    Bioc-devel at r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel
Gordon K Smyth
# 
Hi Martin,

I may have jumped to the wrong conclusion as to the cause of the problem. The problem we are observing is that BiocManager::install does not currently work with R Devel for Windows 2020-04-18 (which the installer is calling R 4.1.0) while it still works fine with R Devel for Windows 2020-02-15 (which the installer is calling R 4.0.0). In the former case, the installer is trying to find a 4.1 repository that doesn't seem to exist.

Note that I have not renamed R-devel to R 4.1.0 myself. The r-devel-win.exe installer automatically labels the R shortcut as "R 4.1.0".

I am assuming that Bioconductor packages that need compilation have to be build using the same toolchain as used for R itself. Since I can install Bioc packages with R 4.0.0, I assume that I am getting packages built with Rtools3.5. If there is an Rtools4.0 version of Bioc-devel, where is it?

I can git clone the source code for Bioc packages and compile them successfully for R 4.1.0 using Rtools4.0, but that's more tedious that using install().

It is getting very close to the Bioc 3.11 Release, so we would like to make sure that everything works ok with the latest version of R.

Regards
Gordon

------------- R Session with R 4.1.0 for Windows  (fails) --------------

R Under development (unstable) (2020-04-18 r78255) -- "Unsuffered Consequences"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
--- Please select a CRAN mirror for use in this session ---
trying URL 'https://mirror.aarnet.edu.au/pub/CRAN/bin/windows/contrib/4.1/BiocManager_1.30.10.zip'
Content type 'application/zip' length 99799 bytes (97 KB)
downloaded 97 KB

package ?BiocManager? successfully unpacked and MD5 sums checked

The downloaded binary packages are in
        C:\Users\smyth\AppData\Local\Temp\RtmpaqaFCD\downloaded_packages
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
Bioconductor version 3.11 (BiocManager 1.30.10), R Under development (unstable) (2020-04-18 r78255)
Installing package(s) 'BiocVersion', 'edgeR'
also installing the dependencies ?limma?, ?locfit?, ?Rcpp?

Warning: unable to access index for repository https://bioconductor.org/packages/3.11/bioc/bin/windows/contrib/4.1:
  cannot open URL 'https://bioconductor.org/packages/3.11/bioc/bin/windows/contrib/4.1/PACKAGES'
Warning: unable to access index for repository https://bioconductor.org/packages/3.11/data/annotation/bin/windows/contrib/4.1:
  cannot open URL 'https://bioconductor.org/packages/3.11/data/annotation/bin/windows/contrib/4.1/PACKAGES'
Warning: unable to access index for repository https://bioconductor.org/packages/3.11/data/experiment/bin/windows/contrib/4.1:
  cannot open URL 'https://bioconductor.org/packages/3.11/data/experiment/bin/windows/contrib/4.1/PACKAGES'
Warning: unable to access index for repository https://bioconductor.org/packages/3.11/workflows/bin/windows/contrib/4.1:
  cannot open URL 'https://bioconductor.org/packages/3.11/workflows/bin/windows/contrib/4.1/PACKAGES'
Packages which are only available in source form, and may need compilation of
  C/C++/Fortran: ?limma? ?edgeR?
  These will not be installed
trying URL 'https://mirror.aarnet.edu.au/pub/CRAN/bin/windows/contrib/4.1/locfit_1.5-9.4.zip'
Content type 'application/zip' length 670513 bytes (654 KB)
downloaded 654 KB

trying URL 'https://mirror.aarnet.edu.au/pub/CRAN/bin/windows/contrib/4.1/Rcpp_1.0.4.6.zip'
Content type 'application/zip' length 3168097 bytes (3.0 MB)
downloaded 3.0 MB

package ?locfit? successfully unpacked and MD5 sums checked
package ?Rcpp? successfully unpacked and MD5 sums checked

The downloaded binary packages are in
        C:\Users\smyth\AppData\Local\Temp\RtmpaqaFCD\downloaded_packages
installing the source package ?BiocVersion?

trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/BiocVersion_3.11.1.tar.gz'
Content type 'application/x-gzip' length 980 bytes
downloaded 980 bytes

* installing *source* package 'BiocVersion' ...
** using staged installation
** help
*** installing help indices
  converting help for package 'BiocVersion'
    finding HTML links ... done
    BiocVersion-pkg                         html
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (BiocVersion)
Making 'packages.html' ... done

The downloaded source packages are in
        ?C:\Users\smyth\AppData\Local\Temp\RtmpaqaFCD\downloaded_packages?

------------- End of R Session with R 4.0.0 for Windows  --------------

------------- R Session with R 4.0.0 for Windows  (successful) --------------

R Under development (unstable) (2020-02-15 r77808) -- "Unsuffered Consequences"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
Error in install("edgeR") : could not find function "install"
+ > library("BiocManager")
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
Bioconductor version 3.11 (BiocManager 1.30.10), R Under development (unstable) (2020-02-15 r77808)
Installing package(s) 'edgeR'
trying URL 'https://bioconductor.org/packages/3.11/bioc/bin/windows/contrib/4.0/edgeR_3.29.1.zip'
Content type 'application/zip' length 3181389 bytes (3.0 MB)
downloaded 3.0 MB

package ?edgeR? successfully unpacked and MD5 sums checked

The downloaded binary packages are in
        C:\Users\smyth\AppData\Local\Temp\Rtmpy8bFkk\downloaded_packages
Old packages: 'backports', 'BiocFileCache', 'BiocNeighbors', 'BiocStyle', 'broom', 'dbplyr',
  'foreign', 'GenomeInfoDb', 'GenomeInfoDbData', 'gtools', 'isoband', 'MSnbase', 'nlme', 'purrr',
  'Rcpp', 'RcppArmadillo', 'RCurl', 'Rhdf5lib', 'Rsubread', 'scran', 'survival', 'tibble',
  'tinytex', 'withr'
Update all/some/none? [a/s/n]: n
------------- R Session with R 4.0.0 for Windows  --------------
_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________
Hervé Pagès
# 
Hi Gordon,

The Bioconductor version to be released (BioC 3.11) uses R 4.0, not R 
4.1. The latest Windows built of R 4.0 is available on CRAN here:

   https://cran.r-project.org/bin/windows/base/rpatched.html

Note that the upcoming devel version of Bioconductor (BioC 3.12) will 
still be based on R 4.0. We'll only start using R 4.1 for BioC 3.13.

Cheers,
H.
On 4/20/20 17:42, Gordon K Smyth wrote:
Gordon K Smyth
# 
Hi Herve,

Ah, I see now.

I was getting confused by the CRAN page that points to R-4.1.0 for Windows and says it "will eventually become the next major release of R". I see now that we should be using R-4.0.0 RC instead, which is the patched link.

Can I confirm whether we should be using Rtools3.5 or Rtools4.0 with R-4.0.0 RC for Windows? The release note for R-4.0.0 RC says that it was ported by Guido Masarotto, Brian Ripley and Duncan Murdoch, which to me means it must be using the older toolchain. I had associated that toolchain in my mind with Rtool3.5, but perhaps without any proper justification. Maybe Rtools4.0 supports both the 4.0.0 and 4.1.0 pipelines.

Thanks
Gordon
_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________
Hervé Pagès
# 
Yes, all builds of R 4.0 and above are now using Rtools40 on Windows. See:

   https://cran.r-project.org/bin/windows/Rtools/

AFAIK there has been no official announcement about this on the various 
R-* mailing lists but this is suggested by the output of 'R CMD config 
CC' which has changed from C:/Rtools/mingw_64/bin/gcc to 
/mingw64/bin/gcc with the most recent Windows builds of R 4.0.0.

/mingw64/bin/gcc and /mingw32/bin/gcc are the paths to the 64-bit and 
32-bit compilers **in the Msys environment** that is shipped with 
Rtools40. These paths are relative to the location of Rtools40, which by 
default is C:\Rtools40.

Hope this helps,
H.
On 4/20/20 19:38, Gordon K Smyth wrote:
Gordon K Smyth
# 
Thanks, that does help. So it seems that R-4.0.0 will use

  rtools40/mingw64/bin/gcc.exe

and I'm guessing that R-4.1.0 uses

  rtools40/mingw64/bin/x86_64-w64-mingw32-gcc-8.3.0.exe

Both files in are in the same directory, so R-4.0.0 and R-4.1.0 should both work with same Windows setup (which is afterall what the Rtools webpage says).

Thanks
Gordon
_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________
Hervé Pagès
#
On 4/20/20 23:28, Gordon K Smyth wrote:
I didn't try R 4.1.0 yet so don't know what it uses but I'm surprised it 
uses something different than R 4.0.0. Are you saying that 'R CMD config 
CC' returns /mingw64/bin/x86_64-w64-mingw32-gcc-8.3.0 with the latest 
Windows build of R 4.1.0?

Anyway I still have 6 months to worry about what R 4.1.0 does ;-)

Cheers,
H.
1 day later
Gordon K Smyth
# 
Hi Herve,
No, it doesn't. I was just trying to interpret the note "This build of R-Devel uses the gcc gcc 8.3.0 toolchain from rtools40." that is attached to R-4.1.0 for Windows on the CRAN download page. There's no similar note attached to R-4.0.0 RC, but I may have jumped to the wrong conclusion.

Gordon
_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________
2 days later
Leonardo Collado Torres
# 
Hi,

This seems related
https://twitter.com/opencpu/status/1253649367946444802?s=21

Best,
Leo
On Wed, Apr 22, 2020 at 6:54 AM Gordon K Smyth <smyth at wehi.edu.au> wrote: