[R-pkg-devel] FW: [CRAN-pretest-archived] CRAN submission RIBMDB 1.0.0

Hello Binit Kumar,

On Wed, 17 Feb 2021 15:12:53 +0000

Error: Ld:/Compiler/gcc-4.9.3/local330/lib
-LD:/RCompile/recent/R/bin/i386 -lR clidriver/bin/db2app64.dll: file
not recognized: file format not recognized

Why is db2app64.dll a non recognized file?

The package is being built for i386 (32-bit) subarchitecture, but the
64-bit dll is being linked to it. I think that the correct way to
identify the target sub-architecture is to inspect the R_ARCH
environment variable from configure.win (in R code, use getenv()).

This is described in WRE 1.2:
<https://cran.r-project.org/doc/manuals/R-exts.html#Configure-and-cleanup>.

Error: ** testing if installed package can be loaded from temporary
location

Error: package or namespace load failed for 'RIBMDB' in
dyn.load(file, DLLpath = DLLpath, ...):

 unable to load shared object
'/srv/hornik/tmp/CRAN/RIBMDB.Rcheck/00LOCK-RIBMDB/00new/RIBMDB/libs/RIBMDB.so':

  libdb2.so.1: cannot open shared object file: No such file or
directory

The bigger problem is trying to manually write under R.home() during
installation. This is not allowed by the CRAN policy and may lead to
problems. (Imagine what would happen if two different packages tried to
write to file.path(R.home(), 'clidriver') and for some reason succeeded
despite it's usually not writable for the end-user on Unix-alikes...)

Instead, you could try to (1) use ./configure to extract the
library to src/ and link to it from there and (2) provide a
src/install.libs.R file to copy libdb2.so from src/ to the same
directory where the package shared object is installed. This still may
be insufficient as you might have to instruct the dynamic linker to look
for libdb2.so in the same directory when the package shared object is
loaded by R; on some compilers this may be done by passing the
-rpath='$ORIGIN' argument to the linker, but this is very
platform-dependent (no idea whether it's going to work on Solaris, for
example).

See WRE 1.1.5 for more info on install.libs.R:
https://cran.r-project.org/doc/manuals/R-exts.html#Package-subdirectories

In short, linking an R package to a binary dependency not provided by a
system package is painful, but not explicitly prohibited by CRAN policy
("Source packages may not contain any form of binary executable code",
and yours downloads binaries when it's built, but doesn't contain any;
"Downloads of additional software <...> as part of package installation
<...> should only use secure download mechanisms", which yours does).

Best regards,
Ivan

Hi Ivan,
Thanks for the response. Appreciate.
But,

  1.  Windows: As I can see in the logs of the machine:

platform =  windows , arch =  x64 , R_Version =  R Under development (unstable) (2021-02-16 r80015)



So that means the architecture on CRAN machine is 64 bit and not 32 bit.

This is extracted by my code using:

platform = .Platform$OS.type

arch = .Platform$r_arch



The dll is selected based on the architecture, it is not hardcoded.

Please let me know if my understanding is wrong.

2)      Linux: I have already used "configure" and that copies the requisite file using "CLIDriver_installer.R" but problem is that R does everything in tmp file and then copy to the library folder of R. As you mentioned, "This still may
be insufficient as you might have to instruct the dynamic linker to look
for libdb2.so in the same directory when the package shared object is
loaded by R; on some compilers this may be done by passing the
-rpath='$ORIGIN' argument to the linker, but this is very
platform-dependent (no idea whether it's going to work on Solaris, for
example)."  So the part that is failing is loading. I want to exclude test loading using "-c -no-test-load". I tested in my local system and it works fine.

Can you suggest how to move ahead?



Also the CRAN upload has a limitation of 100MB. I have some executables for multiple platforms and they together constitute more than 100MB. How to resolve this? Can you please help?




Thanks & Regards,
Binit Kumar
From: Ivan Krylov <krylov.r00t at gmail.com>
Sent: 18 February 2021 01:53
To: Binit Kumar <bkumar at rocketsoftware.com>
Cc: r-package-devel at r-project.org
Subject: Re: [R-pkg-devel] FW: [CRAN-pretest-archived] CRAN submission RIBMDB 1.0.0

EXTERNAL EMAIL



Hello Binit Kumar,

On Wed, 17 Feb 2021 15:12:53 +0000

Error: Ld:/Compiler/gcc-4.9.3/local330/lib
-LD:/RCompile/recent/R/bin/i386 -lR clidriver/bin/db2app64.dll: file
not recognized: file format not recognized

Why is db2app64.dll a non recognized file?

The package is being built for i386 (32-bit) subarchitecture, but the
64-bit dll is being linked to it. I think that the correct way to
identify the target sub-architecture is to inspect the R_ARCH
environment variable from configure.win (in R code, use getenv()).

This is described in WRE 1.2:
<https://cran.r-project.org/doc/manuals/R-exts.html#Configure-and-cleanup<https://cran.r-project.org/doc/manuals/R-exts.html#Configure-and-cleanup>>.

Error: ** testing if installed package can be loaded from temporary
location

Error: package or namespace load failed for 'RIBMDB' in
dyn.load(file, DLLpath = DLLpath, ...):

unable to load shared object
'/srv/hornik/tmp/CRAN/RIBMDB.Rcheck/00LOCK-RIBMDB/00new/RIBMDB/libs/RIBMDB.so':

libdb2.so.1: cannot open shared object file: No such file or
directory

The bigger problem is trying to manually write under R.home() during
installation. This is not allowed by the CRAN policy and may lead to
problems. (Imagine what would happen if two different packages tried to
write to file.path(R.home(), 'clidriver') and for some reason succeeded
despite it's usually not writable for the end-user on Unix-alikes...)

Instead, you could try to (1) use ./configure to extract the
library to src/ and link to it from there and (2) provide a
src/install.libs.R file to copy libdb2.so from src/ to the same
directory where the package shared object is installed. This still may
be insufficient as you might have to instruct the dynamic linker to look
for libdb2.so in the same directory when the package shared object is
loaded by R; on some compilers this may be done by passing the
-rpath='$ORIGIN' argument to the linker, but this is very
platform-dependent (no idea whether it's going to work on Solaris, for
example).

See WRE 1.1.5 for more info on install.libs.R:
https://cran.r-project.org/doc/manuals/R-exts.html#Package-subdirectories<https://cran.r-project.org/doc/manuals/R-exts.html#Package-subdirectories>

In short, linking an R package to a binary dependency not provided by a
system package is painful, but not explicitly prohibited by CRAN policy
("Source packages may not contain any form of binary executable code",
and yours downloads binaries when it's built, but doesn't contain any;
"Downloads of additional software <...> as part of package installation
<...> should only use secure download mechanisms", which yours does).

--
Best regards,
Ivan

================================
Rocket Software, Inc. and subsidiaries ? 77 Fourth Avenue, Waltham MA 02451 ? Main Office Toll Free Number: +1 855.577.4323
Contact Customer Support: https://my.rocketsoftware.com/RocketCommunity/RCEmailSupport
Unsubscribe from Marketing Messages/Manage Your Subscription Preferences - http://www.rocketsoftware.com/manage-your-email-preferences
Privacy Policy - http://www.rocketsoftware.com/company/legal/privacy-policy
================================

This communication and any attachments may contain confidential information of Rocket Software, Inc. All unauthorized use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify Rocket Software immediately and destroy all copies of this communication. Thank you.

On Thu, 18 Feb 2021 02:37:22 +0000

  1.  Windows: As I can see in the logs of the machine:

platform =  windows , arch =  x64 , R_Version =  R Under development
(unstable) (2021-02-16 r80015)

We might need the help of someone else more knowledgeable of Windows
package build process, but here is what I noticed: configure.win calls
Rscript without any environment variables. WRE 1.2 says:

Using ?${R_HOME}/bin/R? rather than just ?R? is necessary in order
to use the correct version of R when running the script as part of R
CMD INSTALL, and the quotes since ?${R_HOME}? might contain spaces.

It might be the case that configure.win ends up calling the wrong
Rscript and causing the problem. It might also be the case that
configure.win should handle both sub-architectures in one run. I'm not
sure. But the error definitely happens while building the package for
the 32-bit sub-architecture, because it says "*** arch - i386" earlier,
and the rest of the messages are consistent with this.

2)      Linux: I have already used "configure" and that copies the
requisite file using "CLIDriver_installer.R" but problem is that R
does everything in tmp file and then copy to the library folder of R.

I think that the correct solution (as long as it's possible) would be
to specify a relative dependency load path when building the package
shared object to avoid hard-coding anything.

Can you suggest how to move ahead?

1. Make ./configure put the dependency into src/
2. Add PKG_LIBS=-Wl,-rpath,'$$ORIGIN' (plus other flags) to
   src/Makevars.in
3. Use install.libs.R to make sure that both package shared object and
   its dependency are installed in the same directory supplied by R
   (that is, file.path(R_PACKAGE_DIR, paste0('libs', R_ARCH)))

This should cover GNU/Linux, Solaris and some BSDs. On macOS,
@loader_path seems to be the magic incantation to be used instead of
$ORIGIN. No idea about linkers on AIX and other platforms you might
prefer to support.

Also the CRAN upload has a limitation of 100MB. I have some
executables for multiple platforms and they together constitute more
than 100MB. How to resolve this? Can you please help?

CRAN source packages aren't allowed to contain executable binaries
anyway, so your solution of downloading the dependencies at build time
from a secure location is as close to following the CRAN policy as it
gets.

Best regards,
Ivan