Hi list, I am working on getting a package <https://github.com/lschneiderbauer/fCWTr> to CRAN. It depends on the FFTW library <https://www.fftw.org/> that is built with single precision support. I am stuck in the submission process and I require your help. Before I come to my questions, some key facts about the package: The package has an autoconf configure script that uses (among other things, like OpenMP checks, etc..) AC_SEARCH_LIBS to check whether required functions of the library 'fftwf' exist, if yes, it adds the corresponding compiler/linker flags; if no, it errs with a descriptive error message. The CRAN Windows as well as Linux build service included fftwf in their fftw build out of the box, and so building there was no problem, R CMD Check passes there. In the past, building for MacOS was more trouble, since its fftw package does not include a single-precision build. I reached out to Simon Urbanek, and he was so kind as to add an appropriate new recipe "fftw-s" that provided an fftw version with single precision support. As of now, R CMD check also passes cleanly on the MacOS build service, thanks to Simon Urbanek's efforts. Now, I am stuck at submission for two reasons: 1. The SystemRequirements specification in the DESCRIPTION file is incorrect. 2. It is said that "the package needs a configure check for fftwf". Add 1. Initially, I had no mention of the "single precision" version of fftw, because I thought it is included everywhere by default. It was stated that I need to add that information. I naturally complied. This is my current version: "SystemRequirements: fftw3 (including single precision support fftw3f), fftw3f_omp (optional), OpenMP (optional)" In the second subscription run, I was told to add "fftw-s" since I require the fftw-s package on MacOS. This does not make much sense to me since "fftw-s" is only the name of this package on Simon Urbanek's MacOS build service. The library file itself is still called fftwf, like it is on any other platform. If I added "fftw-s", I would also need to explain that this is only valid for MacOS which seems to make the SystemRequirements unnecessary verbose. * Can someone explain the reason behind this request to me? * How exactly should I add "fftw-s" to pass the submission process? Add 2. I tried to explain now for the second time in the submission notes, that a check is already in place (see the AC_SEARCH_LIBS paragraph above). But my explanation gets ignored. * What am I doing wrong? * What additional configure checks do I need to add to the package? Thanks a lot for your help! Sincerely, Lukas Schneiderbauer
[R-pkg-devel] SystemRequirements & configure check for FFTW with single precision support
6 messages · Lukas Schneiderbauer, Dirk Eddelbuettel, Ivan Krylov +1 more
Lukas, It is, as you noticed, complicated. One way forward might be to rely on what you can rely on (i.e. a suitable system fftw on two of the three platforms) and to embed and locally build where not. Nowadays a number of packages embedding external libraries and resorting to eg cmake to build locally to provide what their R package needs (pak now rebuilds curl, igraph 'vendors' a number of external libraries, as does duckdb and so on), and so on. It is one approach. Others download if they must (i.e. what I once added to nloptr) but CRAN now leans more against downloads at build time. [1] The problem is that SystemRequirements: is all we have to solve a cross-OS, cross-platform, cross-distro (for Linux), cross-release, ... dependency issue. Something for which a (commonly) single-line of free form text is not all that well suited. But anything more formal would be need to reliably address the cross-product of cpu architecture, operating system, flavour/distro, release, ... Not easy for (less than) a handful of already stretched volunteers. So this is a hard problem; if it could be fixed easily it would have been addressed in the 25+ plus years of CRAN. Cheers, Dirk PS For the smaller subset of three releases for (currently) one architecture, one platform, and one distribution flavor this has been address using the existence of system package manager: r2u covers this reliably and gets you (CRAN) package binaries for Ubuntu with full and complete system dependency integration, see https://eddelbuettel.github.io/r2u -- but it is hard / impossible to generalise to other OSs without a similar package manager.
dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
? Sat, 11 Jan 2025 10:37:20 +0100 Lukas Schneiderbauer <lukas.schneiderbauer at gmail.com> ?????:
I was told to add "fftw-s" since I require the fftw-s package on MacOS.
* Can someone explain the reason behind this request to me?
While the SystemRequirements: field currently contains free-form text, eventually we might be able to evolve it into something machine-readable to install the system-level dependencies automatically. Since the macOS packages are built using the recipes system to provide the dependencies, it makes certain sense to list the name of the recipe for their sake, in addition to the name of the shared library.
* How exactly should I add "fftw-s" to pass the submission process?
I've tried looking for examples using
db <- tools::CRAN_package_db()
subset(db, grepl('recipe|mac', SystemRequirements, ignore.case=TRUE))
...but found nothing relevant. How about something like the following?
SystemRequirements:
fftw3 (including single precision support fftw3f; 'fftw-s' macOS
recipe), fftw3f_omp (optional), OpenMP (optional)
I tried to explain now for the second time in the submission notes, that a check is already in place (see the AC_SEARCH_LIBS paragraph above). But my explanation gets ignored.
I'm sorry for asking this admittedly thick question, but are you sure it's not just R CMD check --as-cran repeating the X-CRAN-Comment: field (which includes the phrase about not testing for single-precision FFTW)? Does the rejection e-mail say "please fix and resubmit", list specific problems, or ask for specific actions?
* What additional configure checks do I need to add to the package?
The configure test for -lfftwf seems fine to me. I wouldn't test for OpenMP's omp.h because testing for a working OpenMP installation is fraught with peril and corner cases [*], but you pick up R's OpenMP flags correctly and have appropriate #ifdef _OPENMP sections in the source code, so that shouldn't cause you any real problems.
Best regards, Ivan [*] https://github.com/Rdatatable/data.table/pull/6642
3 days later
Lukas,
I have not seen the communication so I'm not commenting on that specifically, I only looked at the GitHub link.
Although your configure could be improved (more below), it works well enough to detect fftw3f.
Unfortunately, SystemRequirements don't have a well-defined structure, but there are two commonly used notations:
a) library name and version so in your case that would be something like
libfftw3f (>=3.3.0)
b) deb/rpm package names
libfftw3-dev (deb), fftw3-devel (rpm)
fftw is a bit of a mess, because Debian ships the single-precision static library in libfftw3-dev, but also has libfftw3-single3 which is the dynamic version of the same, while RH does not distinguish. macOS recipe names are usually included only if they are non-obvious, but that would be the case here since installing fftw recipe does not work for you, so mentioning fftw-s is probably a good idea (there are community scripts that try to extract that information from the packages so that the dependencies can be installed).
As for your configure, my main objection would be that it ignores CPPFLAGS (they are not substituted at all even though they *are* used in the tests) and doesn't use pkg-config to get the correct flags. (Also the brew part should go - it's makes unwarranted assumptions [see below] and is entirely superfluous if you use pkg-config instead.) To illustrate what I mean, you get
$ pkg-config --cflags fftw3f
-I/opt/R/x86_64/include
$ pkg-config --libs fftw3f
-L/opt/R/x86_64/lib -lfftw3f -lm
while fCWTr on GitHub only uses
Configuration for fCWTr 0.2.9000
cppflags:
cxxflags:
ldflags:
libs: -lfftw3f
It actually works despite that, because R will inject the other flags for you, but that will only work is fftw is installed in the same location as the other libraries used by R (which is common, but not guaranteed). I wouldn't say it is dealbreaker, but it would recommend it for robustness.
FWIW with homebrew the correct flags are obtainable from pkg-config:
$ pkg-config --libs fftw3f
-L/opt/brew/Cellar/fftw/3.3.8_1/lib -lfftw3f
$ pkg-config --cflags fftw3f
-I/opt/brew/Cellar/fftw/3.3.8_1/include
Cheers,
Simon
On Jan 11, 2025, at 10:37 PM, Lukas Schneiderbauer <lukas.schneiderbauer at gmail.com> wrote: Hi list, I am working on getting a package <https://github.com/lschneiderbauer/fCWTr> to CRAN. It depends on the FFTW library <https://www.fftw.org/> that is built with single precision support. I am stuck in the submission process and I require your help. Before I come to my questions, some key facts about the package: The package has an autoconf configure script that uses (among other things, like OpenMP checks, etc..) AC_SEARCH_LIBS to check whether required functions of the library 'fftwf' exist, if yes, it adds the corresponding compiler/linker flags; if no, it errs with a descriptive error message. The CRAN Windows as well as Linux build service included fftwf in their fftw build out of the box, and so building there was no problem, R CMD Check passes there. In the past, building for MacOS was more trouble, since its fftw package does not include a single-precision build. I reached out to Simon Urbanek, and he was so kind as to add an appropriate new recipe "fftw-s" that provided an fftw version with single precision support. As of now, R CMD check also passes cleanly on the MacOS build service, thanks to Simon Urbanek's efforts. Now, I am stuck at submission for two reasons: 1. The SystemRequirements specification in the DESCRIPTION file is incorrect. 2. It is said that "the package needs a configure check for fftwf". Add 1. Initially, I had no mention of the "single precision" version of fftw, because I thought it is included everywhere by default. It was stated that I need to add that information. I naturally complied. This is my current version: "SystemRequirements: fftw3 (including single precision support fftw3f), fftw3f_omp (optional), OpenMP (optional)" In the second subscription run, I was told to add "fftw-s" since I require the fftw-s package on MacOS. This does not make much sense to me since "fftw-s" is only the name of this package on Simon Urbanek's MacOS build service. The library file itself is still called fftwf, like it is on any other platform. If I added "fftw-s", I would also need to explain that this is only valid for MacOS which seems to make the SystemRequirements unnecessary verbose. * Can someone explain the reason behind this request to me? * How exactly should I add "fftw-s" to pass the submission process? Add 2. I tried to explain now for the second time in the submission notes, that a check is already in place (see the AC_SEARCH_LIBS paragraph above). But my explanation gets ignored. * What am I doing wrong? * What additional configure checks do I need to add to the package? Thanks a lot for your help! Sincerely, Lukas Schneiderbauer [[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
2 days later
Thanks for the feedback and advice! I removed the omp.h test, I guess using AC_OPENMP should do the trick anyways. I will also try your suggestion regarding the SystemRequirements. Let's see if it will be accepted this time. I'm sorry for asking this admittedly thick question, but are you sure
it's not just R CMD check --as-cran repeating the X-CRAN-Comment: field (which includes the phrase about not testing for single-precision FFTW)? Does the rejection e-mail say "please fix and resubmit", list specific problems, or ask for specific actions?
I am pretty confident it was also a personal response, not only the X-CRAN-Comment: field. But let's see how the next run goes. Best regards, Lukas
Hi Simon, Thank you for your detailed feedback! I rebuilt the configure script to use pkg-config, removed the extra brew-check, fixed the CPPFLAGS issue you mentioned, and included "fftw-s" in SystemRequirements. The package is building fine again on all platforms (using rhub tests and the `devtools::check_win_devel()` test). Hopefully it will get accepted soon. :) Best regards, Lukas On Wed, Jan 15, 2025 at 3:59?AM Simon Urbanek <simon.urbanek at r-project.org> wrote:
Lukas,
I have not seen the communication so I'm not commenting on that
specifically, I only looked at the GitHub link.
Although your configure could be improved (more below), it works well
enough to detect fftw3f.
Unfortunately, SystemRequirements don't have a well-defined structure, but
there are two commonly used notations:
a) library name and version so in your case that would be something like
libfftw3f (>=3.3.0)
b) deb/rpm package names
libfftw3-dev (deb), fftw3-devel (rpm)
fftw is a bit of a mess, because Debian ships the single-precision static
library in libfftw3-dev, but also has libfftw3-single3 which is the dynamic
version of the same, while RH does not distinguish. macOS recipe names are
usually included only if they are non-obvious, but that would be the case
here since installing fftw recipe does not work for you, so mentioning
fftw-s is probably a good idea (there are community scripts that try to
extract that information from the packages so that the dependencies can be
installed).
As for your configure, my main objection would be that it ignores CPPFLAGS
(they are not substituted at all even though they *are* used in the tests)
and doesn't use pkg-config to get the correct flags. (Also the brew part
should go - it's makes unwarranted assumptions [see below] and is entirely
superfluous if you use pkg-config instead.) To illustrate what I mean, you
get
$ pkg-config --cflags fftw3f
-I/opt/R/x86_64/include
$ pkg-config --libs fftw3f
-L/opt/R/x86_64/lib -lfftw3f -lm
while fCWTr on GitHub only uses
Configuration for fCWTr 0.2.9000
cppflags:
cxxflags:
ldflags:
libs: -lfftw3f
It actually works despite that, because R will inject the other flags for
you, but that will only work is fftw is installed in the same location as
the other libraries used by R (which is common, but not guaranteed). I
wouldn't say it is dealbreaker, but it would recommend it for robustness.
FWIW with homebrew the correct flags are obtainable from pkg-config:
$ pkg-config --libs fftw3f
-L/opt/brew/Cellar/fftw/3.3.8_1/lib -lfftw3f
$ pkg-config --cflags fftw3f
-I/opt/brew/Cellar/fftw/3.3.8_1/include
Cheers,
Simon
On Jan 11, 2025, at 10:37 PM, Lukas Schneiderbauer <
lukas.schneiderbauer at gmail.com> wrote:
Hi list, I am working on getting a package <
CRAN. It depends on the FFTW library <https://www.fftw.org/> that is
built
with single precision support. I am stuck in the submission process and I require your help. Before I come to my questions, some key facts about the package: The package has an autoconf configure script that uses (among other
things,
like OpenMP checks, etc..) AC_SEARCH_LIBS to check whether required functions of the library 'fftwf' exist, if yes, it adds the corresponding compiler/linker flags; if no, it errs with a descriptive error message. The CRAN Windows as well as Linux build service included fftwf in their fftw build out of the box, and so building there was no problem, R CMD Check passes there. In the past, building for MacOS was more trouble,
since
its fftw package does not include a single-precision build. I reached out to Simon Urbanek, and he was so kind as to add an appropriate new recipe "fftw-s" that provided an fftw version with single precision support. As
of
now, R CMD check also passes cleanly on the MacOS build service, thanks
to
Simon Urbanek's efforts. Now, I am stuck at submission for two reasons: 1. The SystemRequirements specification in the DESCRIPTION file is incorrect. 2. It is said that "the package needs a configure check for fftwf". Add 1. Initially, I had no mention of the "single precision" version of fftw, because I thought it is included everywhere by default. It was stated
that
I need to add that information. I naturally complied. This is my current version: "SystemRequirements: fftw3 (including single precision support fftw3f), fftw3f_omp (optional), OpenMP (optional)" In the second subscription run, I was told to add "fftw-s" since I
require
the fftw-s package on MacOS. This does not make much sense to me since "fftw-s" is only the name of this package on Simon Urbanek's MacOS build service. The library file itself is still called fftwf, like it is on any other platform. If I added "fftw-s", I would also need to explain that
this
is only valid for MacOS which seems to make the SystemRequirements unnecessary verbose. * Can someone explain the reason behind this request to me? * How exactly should I add "fftw-s" to pass the submission process? Add 2. I tried to explain now for the second time in the submission notes, that
a
check is already in place (see the AC_SEARCH_LIBS paragraph above). But
my
explanation gets ignored.
* What am I doing wrong?
* What additional configure checks do I need to add to the package?
Thanks a lot for your help!
Sincerely, Lukas Schneiderbauer
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel