Skip to content

Rebuilding and re-checking of downstream dependencies on CRAN Mac build machines

2 messages · Simon Urbanek, Winston Chang

Original
Simon Urbanek
# 
[moving to R-SIG-Mac where it belongs]

Winston,

the error in the issue 
https://github.com/rstudio/httpuv/issues/260
seems to come from libuv itself - possibly a compatibility issue with the latest Apple Security framework.

The issue in 
https://github.com/RcppCore/Rcpp/issues/1046
is genuine and a bug in Rcpp as pointed out by Kevin. My understanding is that it was merged (I was following it on Rcpp mailing list and it looked that it's taken care of).

Please note that another important factor is SDK used - the CRAN El-Capitan builds are done against the 10.11 SDK:

$ ls -l /Library/Developer/CommandLineTools/SDKs/
total 8
lrwxr-xr-x  1 root  wheel   15  6 Apr  2019 MacOSX.sdk -> MacOSX10.11.sdk
drwxr-xr-x  5 root  wheel  170  7 Aug  2017 MacOSX10.11.sdk

The R 4.0.0 builds will be done against the 10.13 SDK. You can check against later SDKs to make sure your package doesn't break on deprecated features, but generally, the CRAN builds are conservative to make sure the binaries run on older systems since not all users upgrade to the latest macOS (for a very good reason as we've seen with the carnage caused by Catalina). 

To make it very clear, CRAN's goal is not to detect incompatibilities with latest SDK, but rather to provide stable binaries, so if a package breaks outside of CRAN, it is wise for the package developers to fix such issues, but fortunately they don't affect the availability on CRAN. That is also one of the reasons why CRAN is more stable than the 3rd party package managers.

Cheers,
Simon
Winston Chang
# 
You're right that the error originally reported in
https://github.com/rstudio/httpuv/issues/260 wasn't due to Rcpp. When I was
investigating the issue, I encountered a different error (the output of
which I posted to
https://gist.github.com/wch/c70b438381c9d2a8b1f917b054e0ba7e) which was
caused by Rcpp -- it's the same uuid_t/uid_t error that was seen in
https://github.com/RcppCore/Rcpp/issues/1046. More details on that below if
anyone is interested.

Regarding the bug from https://github.com/RcppCore/Rcpp/issues/1046, the
fix was merged, but it has not been released to CRAN.

It sounds like perhaps the 10.11 SDK may be a factor. I'll have to check
whether any of us who have tried to build these various packages have been
using the 10.11 SDK.

===========================

This is how I tested that Kevin's fix from
https://github.com/RcppCore/Rcpp/pull/1047 fixed my httpuv problem. I
installed the commit with the fix, and then installed httpuv from CRAN, and
that works. Then I installed the parent commit that Kevin branched from,
and then tried to install httpuv from CRAN, but that failed.

Here are the commands I used to test:
  # Installing Kevin's fix from Rcpp #1047 works
  remotes::install_github("RcppCore/Rcpp at 46bf2e2")
  install.packages("httpuv", type = "source")

  # Doesn't work (this is the parent of the 46bf2e2)
  remotes::install_github("RcppCore/Rcpp at 68a5489")
  install.packages("httpuv", type = "source")  # Results in the
uuid_t/uid_t errors

-Winston


On Thu, Mar 26, 2020 at 5:07 PM Simon Urbanek <simon.urbanek at r-project.org>
wrote:
latest Apple Security framework.
that it was merged (I was following it on Rcpp mailing list and it looked
that it's taken care of).
El-Capitan builds are done against the 10.11 SDK:
against later SDKs to make sure your package doesn't break on deprecated
features, but generally, the CRAN builds are conservative to make sure the
binaries run on older systems since not all users upgrade to the latest
macOS (for a very good reason as we've seen with the carnage caused by
Catalina).
with latest SDK, but rather to provide stable binaries, so if a package
breaks outside of CRAN, it is wise for the package developers to fix such
issues, but fortunately they don't affect the availability on CRAN. That is
also one of the reasons why CRAN is more stable than the 3rd party package
managers.
wrote:
/Users/winston/R/3.6/BH/include/boost/optional.hpp:15:
/Users/winston/R/3.6/BH/include/boost/config.hpp:57:
/Users/winston/R/3.6/BH/include/boost/config/detail/posix_features.hpp:18:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:664:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uid_t.h:31:31:
build+check
(it would take over 24h - currently the nightly build of packages takes
16-22 hours), but we're currently building a new setup for R 4.0.0 on new
hardware and as a part of it we are planning to setup a "mac-builder"
similar to what is currently available for Windows.
(against Rcpp 1.0.4) and I saw no issues. I have seen the discussion on
Rcpp, but so far no one actually posted details of what is breaking (nor do
your links include any actual details on this). I'd love to help, but the
lack fo a useful report makes this impossible. If you have any actual
leads, please post them. The CRAN machine uses the tools that are available
on CRAN: https://cran.r-project.org/bin/macosx/tools/ (clang-7 and
gfortran-6.1 for 3.6.x)
wrote: