R problems with lapack with gfortran
On 5/6/19 12:57 PM, Janne Blomqvist wrote:
On Mon, May 6, 2019 at 11:55 AM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
On 5/4/19 6:49 PM, Steve Kargl wrote:
On Sat, May 04, 2019 at 06:42:47PM +0200, Thomas K?nig wrote:
- figure out Fortran2003 specification for C/Fortran interoperability -- this _sounds_ like the right solution, but I don't think many understand how to use it and what is implied (in particular, will it require making changes to LAPACK itself?)
That would actually be fairly easy. If you declare the subroutines
BIND(C), as in
subroutine foo(a,b) BIND(C,name="foo_")
real a
character*1 b
end
you will get the calling signature that you already have in your C
sources.
This also has the advantage of being standards compliant, and would be
probably be the preferred method.
With the caveat that one may need to use the VALUE attribute to account for pass-by-value vs pass-by-reference.
This seems clean solution, but as I said before not easy, because currently the tradition is to call the Fortran interface directly from C (not via any C wrappers). This means one could not substitute LAPACK/BLAS at dynamic linking time, unless all LAPACK/BLAS implementations agreed on such a C interface. Now the substitution is based on the original Fortran interface. In case of R, if we only used the included reference BLAS/LAPACK, we could do this, define our wrappers, say "c_dgemm" for "dgemm", change R to call via that interface, ask maintainers of all packages to change their code to call via their interface, and this should work with all Fortran 2003 compilers. But, R is often used also with optimized BLAS/LAPACK implementations that can be substituted at dynamic linking time. And there we could do nothing at R level to help: we cannot generate such wrappers for an existing LAPACK/BLAS implementation (we don't have the source code, the compiler, etc). It would be certainly a good thing if BLAS/LAPACK, with all implementation and uses, switched to a way that is compliant with current Fortran standard. But this should best start with the reference BLAS/LAPACK, continue with other BLAS/LAPACK implementations, and then with systems using those libraries, including R and its packages. Unless/before this happens, it would really be great if we could still use gfortran to build and use this fundamental software library.
Hi, I don't think modifying your own builtin LAPACK makes sense, as you mention that would make R incompatible with another LAPACK provided by the system. And modifying LAPACK upstream by adding BIND(C) wouldn't work either, as that would break all the existing Fortran code calling LAPACK (as LAPACK is F77-style implicit interfaces, the Fortran caller has no knowledge of the interface and thus it must match the compiler default Fortran ABI).
I meant creating another layer of functions using BIND(C), which will have different names (e.g. c_dgemm), and call into the original Fortran functions (e.g. dgemm). Fortran programs could still call into the original Fortran functions. Something like c_dgemm in Chapter 10.2 of "Numerical Computing with Modern Fortran" by Richard J. Hanson and Tim Hopkins, the code is available from https://archive.siam.org/books/ot134/Chapter10/index.php. So LAPACK/BLACK implementations won't break their applications by adding these new functions. Also indeed it would be useful to have C prototypes matching those interfaces distributed with BLAS/LAPACK.
So the remaining place where this could be fixed would be in your C prototypes for LAPACK functions, so that they match what LAPACK expects. Arguably that's the correct approach anyway. I suppose for this task extending the GFortran -fc-prototypes option to generate C prototypes for external functions as well would help?
Well but the non-BIND(C) interface is different for different Fortran compilers, and it changed even between gfortran versions (the type of the lengths). So we would not be able to switch BLAS/LAPACK implementations anymore at dynamic linking time. We would still have to talk to all R package maintainers that manage packages that call directly to BLAS/LAPACK. I still think -fc-prototypes would be useful, certainly it should work for all BIND(C) procedures (the 20190426 does not for the dgemm example from the book), and it would be useful even for non-BIND(C) procedures.
AFAICS, this interface mismatch problem affects other Fortran compilers as well, just that by sheer luck this has worked mostly so far (that is, other Fortran compilers also expect a hidden string length argument, with no exception for length==1 strings). But with increasingly sophisticated interprocedural optimizations such sins can no longer be forgiven.
Best Tomas