Skip to content

[R-pkg-devel] Package builds on all systems except on Fedora with clang

5 messages · Pepijn de Vries, Ivan Krylov, Dirk Eddelbuettel

Original
Pepijn de Vries
# 
Dear fellow developers,

Recently I've submitted a package to CRAN which builds without problems on most systems:

https://cran.r-project.org/web/checks/check_results_openmpt.html

Except for Fedora with clang:

https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-clang/openmpt-00install.html

There are some valgrind and ASAN issues, but most of them are solved at https://github.com/pepijn-devries/openmpt

At first I thought that the problem was caused by a missing system requirement (i.e., the libopenmpt library). However, Prof. Ripley reported that all system requirements are installed on the Fedora machine. He also mentioned that I "need to check in your configure script that the external library you need is available *and* usable with the compilers used to compile R and packages." As I'm new to including static libraries in an R package, I'm not sure how to fix this. Does anyone have a clue to get this fixed?

Kind regards,

Pepijn
Ivan Krylov
# 
? Sun, 19 Jan 2025 17:07:51 +0000
Pepijn de Vries <pepijn.devries at outlook.com> ?????:
The Fedora-clang test fails with a symbol lookup error:
Translated using 'c++filt', the missing function turns out to be
openmpt::module::ctl_get_integer(std::__1::basic_string_view<char,
std::__1::char_traits<char> >) const. Why would libopenmpt.so be
missing a method for an std::string_view?

The problem here is that the Clang checks compile C++ code with the
"libc++" standard library that belongs to the LLVM project (same people
who develop Clang). The openmpt library, on the other hand, comes from
Fedora and is compiled and linked with the GNU "libstdc++" standard
library. The two standard libraries have very different internals and
can't be mixed. There's nothing you should do as a package developer to
make it work [1] (thank you Tomas for the clarification!).

I think that Prof. Ripley is asking you to make a configure test that
fails in these circumstances. Here's one that should work:

1. Write a C++ source file that includes OpenMPT headers and exports a
single function that calls the problematic OpenMPT function.

2. During ./configure, export the environment variables PKG_CPPFLAGS,
PKG_LIBS with the same contents that you intend to give to Makevars.

3. From the same ./configure script, call ${R_HOME}/bin/R CMD SHLIB to
compile the source file from (1) and link it into a shared library.

4. Call R again to dyn.load() the shared library. (Use
.Platform$dynlib.ext to figure out the file extension.)
Pepijn de Vries
# 
Hi Ivan,

Thank you for the quick response. The linked discussion thread was also helpful.

I think I could write a similar test as used by `cpp11tesseract`:

https://github.com/pachadotdev/cpp11tesseract/blob/2ea8287ef2c27901446bafa402728014d99904d4/configure#L66-L85

Kind regards,

Pepijn

________________________________________
Van:?Ivan Krylov <ikrylov at disroot.org>
Verzonden:?zondag 19 januari 2025 20:40
Aan:?Pepijn de Vries <pepijn.devries at outlook.com>
CC:?Ivan Krylov via R-package-devel <r-package-devel at r-project.org>
Onderwerp:?Re: [R-pkg-devel] Package builds on all systems except on Fedora with clang
?
? Sun, 19 Jan 2025 17:07:51 +0000
Pepijn de Vries <pepijn.devries at outlook.com> ?????:
The Fedora-clang test fails with a symbol lookup error:
Translated using 'c++filt', the missing function turns out to be
openmpt::module::ctl_get_integer(std::__1::basic_string_view<char,
std::__1::char_traits<char> >) const. Why would libopenmpt.so be
missing a method for an std::string_view?

The problem here is that the Clang checks compile C++ code with the
"libc++" standard library that belongs to the LLVM project (same people
who develop Clang). The openmpt library, on the other hand, comes from
Fedora and is compiled and linked with the GNU "libstdc++" standard
library. The two standard libraries have very different internals and
can't be mixed. There's nothing you should do as a package developer to
make it work [1] (thank you Tomas for the clarification!).

I think that Prof. Ripley is asking you to make a configure test that
fails in these circumstances. Here's one that should work:

1. Write a C++ source file that includes OpenMPT headers and exports a
single function that calls the problematic OpenMPT function.

2. During ./configure, export the environment variables PKG_CPPFLAGS,
PKG_LIBS with the same contents that you intend to give to Makevars.

3. From the same ./configure script, call ${R_HOME}/bin/R CMD SHLIB to
compile the source file from (1) and link it into a shared library.

4. Call R again to dyn.load() the shared library. (Use
.Platform$dynlib.ext to figure out the file extension.)

--
Best regards,
Ivan

[1] https://stat.ethz.ch/pipermail/r-package-devel/2024q4/011326.html
Ivan Krylov
# 
? Sun, 19 Jan 2025 20:42:17 +0000
Pepijn de Vries <pepijn.devries at outlook.com> ?????:
I should have replied to that thread too, but got swamped. If the
compiler flags are right, $CXX -c conftest.cpp will succeed the same way
that the individual object files currently successfully compile on
Fedora-clang [1]:
Even linking the shared library is not enough, because that step
succeeds on Fedora-clang too:
Testing only the previous two steps will succeed during ./configure on
the Fedora-clang check and then fail to load the shared library during
package installation, like it currently does. Using R CMD SHLIB, on the
other hand, spares the effort of trying to figure out the right
compiler and giving it all the right flags.

Unrelated, but speaking of the default linker flags,
These are a bit counter-intuitive. When you ask the linker to link with
-lfoo, it tries to link with libfoo.a or libfoo.so, i.e. it prepends
the "lib" by itself.

Also unrelated, but is there OpenMPT in the macOS 'recipes' system? I
think it's the recommended way of making use of third-party code in
CRAN packages [2].
Dirk Eddelbuettel
#
On 19 January 2025 at 20:42, Pepijn de Vries wrote:
| I think I could write a similar test as used by `cpp11tesseract`:
| 
| https://github.com/pachadotdev/cpp11tesseract/blob/2ea8287ef2c27901446bafa402728014d99904d4/configure#L66-L85

Taking an example from a package not on CRAN 'for policy violations' may not
be the safest best.

`autoconf` has included the ability to conduction these tests for a long
time. Here is an (old) snippet from RProtoBuf which uses a test file to
assert we have a recent enough version:

   ## also check for minimum version
   AC_MSG_CHECKING([if ProtoBuf version >= 2.2.0])
   AC_RUN_IFELSE([AC_LANG_SOURCE([[
   #include <google/protobuf/stubs/common.h>
   int main() {
      if (GOOGLE_PROTOBUF_VERSION >= 2001000) {
           exit (0);
      } else {
           exit(1);
      }
   }
   ]])],
   [pb_version_ok=yes],
   [pb_version_ok=no],
   [pb_version_ok=yes])
   if test x"${pb_version_ok}" = x"no"; then
       AC_MSG_ERROR([Need ProtoBuf version >= 2.2.0])
   else
       AC_MSG_RESULT([yes])
   fi

Note that this happens after we found suitable compiler and linker switches.
And of course, switching to `autoconf` is no small task either. But getting
an external library to build reliably on all platforms is one of the harder
things to set up at CRAN.

Dirk