Skip to content

Enable compression in download.file()

1 message · Jeroen Ooms

#
Comparing download times of base R and the curl package:

  url <- 'https://jeroen.github.io/data/nycflights13.json'
  tmp <- tempfile()
  system.time(download.file(url, tmp, quiet = F)
  system.time(curl::curl_download(url, tmp, quiet = F))

Because this is an uncompressed file, the gzipped download is about 7x faster.

For the solution see https://curl.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html :