Skip to content

[R-pkg-devel] Issue with R Package on CRAN - OpenMP and clang17

6 messages · Romain Pierlot, Ivan Krylov, Dirk Eddelbuettel +2 more

Romain Pierlot
# 
Hi there, 

I'm sorry to disturb, I hope I send a mail at the right place. 





I have been encountering a challenge with an R package that I've been working on, which was recently removed from CRAN due to some issues in Fortran. 

It appears that I have addressed the Fortran issues, but upon resubmission to CRAN, the package was not accepted, and the following error was returned : 


''With clang17 it still fails to install: 

error: loc("/data/gannet/ripley/R/packages/incoming/frailtypack.Rcheck/00_pkg_src/frailtypack/src/Integrale_mult_scl.f90":1811:17): 'omp.reduction' op must be used within an operation supporting reduction clause interface 
error: verification of lowering to FIR failed 

LLVM trunk gives the same error. 
'' 




I have attached the portion of the code where this issue is reported. From my understanding, it appears to be related to parallelization, but I have been unable to resolve it on my own. 

I would greatly appreciate any assistance or advice that you might be able to offer. If you require any additional information, please do not hesitate to let me know. 




Moreover, I don't know if it is normal or not, but when I check my package with R CMD chech --as-cran, I don't have any error or note message. But when I submit the package to the CRAN, they return errors and notes. 








Thank you very much for your time and consideration. 




Best regards, 




Romain Pierlot
Ivan Krylov
# 
? Mon, 30 Oct 2023 15:58:31 +0100 (CET)
Romain Pierlot <romain.pierlot at u-bordeaux.fr> ?????:
This is exactly the right place to ask. Welcome to R-package-devel!
I'm not 100% sure, but I currently think that this is a compiler bug.

You have a variable, ss, declared inside a function.

If nb_procs==1, you start an OpenMP parallel region with a reduction on
that variable. The OpenMP parallel region starts and ends inside that
if clause. Outside the parallel region, in another branch, you add to
it as usual.

Flang-new complains about your use of the 'ss' variable outside the
parallel region, alleging that it's not valid for an OpenMP reduction.
There is no OpenMP reduction in sight where the error is reported. I
think that Flang-new is wrong to signal an error here.

Unfortunately, proving that this is a compiler bug might be hard. I'll
try to reproduce your problem, starting by downloading and compiling
flang from https://github.com/llvm/llvm-project.git commit
092b6c5ee3707ea10b9f10d0a674e8d12395369b (as stated at
<https://www.stats.ox.ac.uk/pub/bdr/clang17/frailtypack.log>). I hope
it will be possible to simplify Integrale_mult_scl.f90 until it looks
manageable for their bug tracker.
These errors only happen with the "flang-new" Fortran compiler that's
part of the LLVM compiler infrastructure. You're probably using
gfortran or some other, more well-polished Fortran compiler.
1 day later
Ivan Krylov
# 
? Mon, 30 Oct 2023 18:52:29 +0300
Ivan Krylov <krylov.r00t at gmail.com> ?????:
Wow. Building LLVM is an adventure, and I don't mean it in a good way.
I had to patch flang/include/flang/Semantics/symbol.h because it
declares `friend class std::array` while my copy of the C++ standard
library has a `struct std::array`, which makes the friend declaration
an error. Naively thinking that I would be helping debug the problem, I
tried compiling a debugging build of LLVM, only to watch in horror as
it consumed all disk space I could offer it. I eventually ran out of
things to delete after the build directory reached 100 gigabytes in
size, while the build still had 1000 out of 6000 targets to compile and
link. Even the release build failed at 410 remaining targets because of
a missing -fPIC somewhere in the compiler command line. Thankfully, it
still produced a working `flang-new` executable.

A release build of LLVM + flang-new takes a relatively reasonable 
8.2 GBytes of storage. The computers that helped launch the first
people into space had 2 kWords of memory, but nowadays you need more
than 256 MBytes of RAM to launch a bird into a pig and 10 GBytes of
storage in order to compile a compiler. This is what progress looks
like.

I was able to reduce the example to the following program:

program main
    implicit none

    real :: sum = 0
    integer :: i, n = 10

    sum = sum + 1. ! <-- error here if followed by OpenMP reduction

    !$OMP PARALLEL DO default(none) PRIVATE (i) SHARED(n) &
    !$OMP REDUCTION(+:sum) SCHEDULE(Dynamic,1)
    do i=1,n
        sum = sum + real(i)
    end do
    !$OMP END PARALLEL DO

    print *, sum
end

% flang-new -c -o /dev/null src.f90 -fopenmp
error: loc("src.f90":7:5): 'omp.reduction' op must be used within an
operation supporting reduction clause interface
error: verification of lowering to FIR failed

gfortran raises no errors or warnings for the test program. Bug
reported at <https://github.com/llvm/llvm-project/issues/70828>.

Would that be enough proof of a compiler bug, or do we need a
confirmation from the developers?
Dirk Eddelbuettel
#
On 31 October 2023 at 19:58, Ivan Krylov wrote:
| [...] The computers that helped launch the first
| people into space had 2 kWords of memory, but nowadays you need more
| than 256 MBytes of RAM to launch a bird into a pig and 10 GBytes of
| storage in order to compile a compiler. This is what progress looks
| like.

Fortune!!

Dirk
Jeff Newmiller
# 
ROFL! Seconded!

The quote itself has a bit of greybeard smell to it, but the "ran out of stuff to delete at 100GB with 1000 out of 6000 targets compiled" had me in stitches.
On October 31, 2023 10:16:10 AM PDT, Dirk Eddelbuettel <edd at debian.org> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Greg Hunt
    

    

      
      #
    

  


  

      
If i remember rightly one of the early Algol compilers for the IBM
mainframe couldnt be compiled on an IBM mainframe because it was too memory
hungry (it had to be cross compiled). The numbers change, but the problems
don?t, except that i haven?t run a compile lately that ran out of memory
like that.

        
On Tue, 31 Oct 2023 at 12:24 pm, Dirk Eddelbuettel <edd at debian.org> wrote: