Libre SSL bug on MacOS Monterey => error in download.file()
On Jan 13, 2022, at 12:47 AM, Jeroen Ooms <jeroenooms at gmail.com> wrote: On Tue, Jan 11, 2022 at 10:12 PM Simon Urbanek <simon.urbanek at r-project.org> wrote:
Pet?e, thanks, for the detailed analysis. It is rather curious that the issue appears only on _newer_ systems - we are more used to issues due to older CA chains and similar. It looks like an Apple bug on specific systems, so hopefully it will be fixed eventually. In general I was trying to avoid having to supply our own SSL library since that opens a whole can of worms - on one hand due the dependency issues (which libraries get compiled against what) and on the other hand we become responsible for security updates. Thanks to Jeroen for the work-around (CURL_SSL_BACKEND=SecureTransport), using the native API is certainly preferred, there have been several issues with both OpenSSL and LibreSSL before. It seems that Apple has been flip-flopping with libcurl a lot - on El Capitan it was shipped with SecureTransport, on High-Sierra with LibreSSL, on Catalina and higher with both, but Libre the default. I am somewhat less apprehensive to use static libcurl for R than SSL libraries as the fallout is a bit smaller. As a trial I have added static curl[2] which is close to the Apple build minus MultiSSL to big-sur nightly builds of R[3] and as expected that solves the problem. It may not be entirely unproblematic for package space, because packages often forget to prepend --static when using static builds of libraries, and so do other dependencies that may use curl, but I'll see what comes out of it.
I would much recommend to stick with the apple version of libcurl; perhaps override the default ssl-backend if you like. There is some example code to do this in the curl package that you could adapt for base r: https://github.com/jeroen/curl/blob/master/src/ssl.c The benefit of dynamically linking to apple's libcurl is that we automatically get a version of libcurl+deps+certs that is tuned and maintained for that version of macos, including future ones. If you ship a version of base-R with a static libcurl now, that version of R may not work anymore a few years from now or on a future version of macos, when things have moved on (for example, when servers start to require TLS1.3).
Yes, but if you are using an old version of R on a new system, you have a lot of other worries - you can't expect new technologies to work with old software. CURL itself has fewer evolution issues than SSL libraries. As I said, I am a big proponent of re-using system libraries as much as possible, but, for example, High Sierra doesn't ship with ST back-end support, so using a static version that does is better there as Apple doesn't not maintain the curl CAs but it does the system ones so it's arguably better. The current issue is quite curious since breaking on the latest system is quite unusual, just preferring ST works only because it is the latest system that breaks and it has the ST option.
As Brian pointed out static curl has its own issues since its pkg-config flags are broken - that's why I have not activated the add-on recipes by default, I have seen those issues before.
For R itself there are thee options:
a) add CURL_SSL_BACKEND=${CURL_SSL_BACKEND-'SecureTransport'} to $R_HOME/etc/Renviron of the distribution
b) add something like your https://github.com/r-devel/r-svn/pull/75/commits/79b22b461e527e8a46de84c145e8e5fb59e75d14 to R
c) build against static libcurl
The big advantage of the first one is that it applies to all processes, so even command line curl will then work and so will all packages.
The drawback of the second one is that it only applies the R itself. The third one could be done both for R and packages, but causes headaches resp. requires slight patching of libcurl.pc. The advantage is that it can bring more recent curl to all older systems.
I don't have a strong opinion. I am not thrilled with option b) because that is a hack just to react to something which is never a good idea from maintenance point of view (we would require all curl-based code to use it). So I think a) and c) are more palatable with a) having the benefit of handling non-R cases. A slight benefit of c) is that some dependencies require more recent curl version than provided by older systems, so that would cover it at the cost of maintaining the curl binaries. Finally, the real benefit of c) is that if Apple screws things up even more we don't care - we may not be at that point yet, though.
Cheers,
Simon