Some suggested changes to Windows makefiles with respect to experimental toolchain
Hello.
While trying to build WITH LTO, I am seeing core R building properly,
seeing the variables I created (including finding the BLAS) and
passing the '-flto' flag properly up to building lapack.dll. From
"building package 'base'" and on (such as stats, compiler, the
recommended packages), the calls to gcc (compilation and linking) no
longer have -flto in them. However, EOPTS does seem to propogate,
which is why I missed this the first time around as I passed -flto in
EOPTS. Of course, that misses the point as the linker doesn't use the
EOPTS flag, which is why I modified the CC, FC, F77 etc. calls. Which
file is it that controls the calls to gcc and gfortran from building
base and on? If it helps, below is the patch I am using (well,
actually the diff, as Windows doesn't play nicely with "patch").
Thank you very much,
Avi
-------------8<-----------------
diff -ruN C:/R/ORIG/R-patched/src/extra/blas/Makefile.win
C:/R/R-patched/src/extra/blas/Makefile.win
--- C:/R/ORIG/R-patched/src/extra/blas/Makefile.win 2015-04-09
16:34:55.000000000 -0400
+++ C:/R/R-patched/src/extra/blas/Makefile.win 2015-09-08
17:03:04.925632500 -0400
@@ -8,11 +8,11 @@
$(CC) -O3 -I../../include -c $< -o $@
## Rblas.dll imports xerbla_ from R.dll
-ifeq "$(USE_ATLAS)" "YES"
+ifeq "$(USE_EXTBLAS)" "YES"
../../../$(BINDIR)/Rblas.dll: blas00.o ../../gnuwin32/dllversion.o
@$(ECHO) -------- Building $@ --------
$(DLL) -shared $(DLLFLAGS) -o $@ $^ Rblas.def \
- -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lf77blas -latlas
+ -L../../../$(IMPDIR) -lR -L"$(EXTBLAS_PATH)" $(EXTBLASLIBS)
else
../../../$(BINDIR)/Rblas.dll: blas.o cmplxblas.o ../../gnuwin32/dllversion.o
@$(ECHO) -------- Building $@ --------
diff -ruN C:/R/ORIG/R-patched/src/gnuwin32/Makefile
C:/R/R-patched/src/gnuwin32/Makefile
--- C:/R/ORIG/R-patched/src/gnuwin32/Makefile 2015-04-19
11:02:02.000000000 -0400
+++ C:/R/R-patched/src/gnuwin32/Makefile 2015-09-08 17:21:42.945036900 -0400
@@ -65,6 +65,10 @@
DLLFLAGS = -s
endif
+ifdef USE_LTO
+ DLLFLAGS += -Wl,--allow-multiple-definition
+endif
+
R-DLLFLAGS = -mwindows
LIBEXTRAS = -L"$(EXT_LIBS)"/lib$(R_ARCH) -lpcre -lz -lbz2 -llzma
diff -ruN C:/R/ORIG/R-patched/src/gnuwin32/MkRules.dist
C:/R/R-patched/src/gnuwin32/MkRules.dist
--- C:/R/ORIG/R-patched/src/gnuwin32/MkRules.dist 2015-04-19
11:02:01.000000000 -0400
+++ C:/R/R-patched/src/gnuwin32/MkRules.dist 2015-09-08
17:07:23.112662800 -0400
@@ -26,9 +26,12 @@
# an alternative is to use -gstabs here, if the debugger supports only stabs.
# G_FLAG = -gdwarf-2
-# Set to YES and specify the path if you want to use the ATLAS BLAS.
-# USE_ATLAS = NO
-# ATLAS_PATH =
+# Set to YES and specify the path if you want to use a pre-compiled
ATLAS or OpenBLAS EXTERNAL BLAS.
+# USE_EXTBLAS = NO
+# EXTBLAS_PATH =
+# Use '-lf77blas -latlas' ast the value for EXTBLASLIBS for ATLAS.
+# Use the appropriate version of -lopenblas for OpenBLAS
+# EXTBLASLIBS =
# Support for the ACML and Goto BLASes has been withdrawn: see R-admin.html
@@ -48,6 +51,8 @@
# Include the trailing /, and use / not \.
# Do this in the more detailed options below
+# To enable link-time optimization using the 4.9.2 toolset, set to YES
+# USE_LTO = YES
### BEGIN more detailed options
# Some of the toolchains have prefixes for e.g. ar, gcc.
diff -ruN C:/R/ORIG/R-patched/src/gnuwin32/MkRules.rules
C:/R/R-patched/src/gnuwin32/MkRules.rules
--- C:/R/ORIG/R-patched/src/gnuwin32/MkRules.rules 2015-04-19
11:02:03.000000000 -0400
+++ C:/R/R-patched/src/gnuwin32/MkRules.rules 2015-09-08
17:09:36.373309600 -0400
@@ -5,10 +5,12 @@
LOCAL_SOFT ?= $(R_HOME)/extsoft
EXT_LIBS ?= $(LOCAL_SOFT)
G_FLAG ?= -gdwarf-2
-USE_ATLAS ?= NO
-ATLAS_PATH ?=
+USE_EXTBLAS ?= NO
+EXTBLAS_PATH ?=
+EXTBLASLIBS ?=
MULTI ?=
TOOL_PATH ?=
+USE_LTO ?=
BINPREF ?=
BINPREF64 ?= x86_64-w64-mingw32-
M_ARCH ?=
@@ -67,6 +69,15 @@
SYMPAT = 's/^.* [BCDRT] _/ /p'
endif
+## Link-time optimization
+ifeq "$(USE_LTO)" "YES"
+LTOPREF = gcc-
+LTOCALL = -flto
+else
+LTOPREF =
+LTOCALL =
+endif
+
## -std=c99 would force __STRICT_ANSI__ disabling strdup etc, and also alloca
C99FLAG=-std=gnu99
@@ -81,10 +92,10 @@
IMPDIR=$(BINDIR)
endif
-AR=$(BINPREF)ar
+AR=$(BINPREF)$(LTOPREF)ar
# AWK=gawk
CAT=cat
-CC=$(BINPREF)gcc$(GCC4_SUFF) $(C99FLAG) $(M_ARCH)
+CC=$(BINPREF)gcc$(GCC4_SUFF) $(C99FLAG) $(M_ARCH) $(LTOCALL)
CP=cp
# as set by make
CPP=$(CC) -E
@@ -94,14 +105,14 @@
## even for native builds
DLLTOOLFLAGS=--as $(BINPREF)as $(DT_ARCH) -k
ECHO=echo
-F77=$(BINPREF)gfortran$(GCC4_SUFF) $(M_ARCH)
+F77=$(BINPREF)gfortran$(GCC4_SUFF) $(M_ARCH) $(LTOCALL)
FLIBS=-lgfortran -lquadmath
LINKER=$(MAIN_LD)
MAIN_LD=$(CC)
MAKEINFO = $(TEXI2ANY)
MKDIR=mkdir
-NM=$(BINPREF)nm
-RANLIB=$(BINPREF)ranlib
+NM=$(BINPREF)$(LTOPREF)nm
+RANLIB=$(BINPREF)$(LTOPREF)ranlib
RESCOMP=$(BINPREF)windres $(RC_ARCH)
# as set by make
diff -ruN C:/R/ORIG/R-patched/src/gnuwin32/fixed/Makefile
C:/R/R-patched/src/gnuwin32/fixed/Makefile
--- C:/R/ORIG/R-patched/src/gnuwin32/fixed/Makefile 2015-04-19
11:02:01.000000000 -0400
+++ C:/R/R-patched/src/gnuwin32/fixed/Makefile 2015-09-08
18:09:11.695519700 -0400
@@ -34,6 +34,8 @@
-e "s|R_ARCH =|R_ARCH = $(R_ARCH)|" \
-e "s|DT_ARCH =|DT_ARCH = $(DT_ARCH)|" \
-e "s|RC_ARCH =|RC_ARCH = $(RC_ARCH)|" \
+ -e "s|LTOCALL =|LTOCALL = $(LTOCALL)|" \
+ -e "s|LTOPREF =|LTOPREF = $(LTOPREF)|" \
-e "s|M_ARCH =|M_ARCH = $(M_ARCH)|" \
-e "s|@SYMPAT@|$(SYMPAT)|" \
-e "s|(TCL_HOME)/bin|(TCL_HOME)/bin64|" \
@@ -49,6 +51,8 @@
-e "s|R_ARCH =|R_ARCH = $(R_ARCH)|" \
-e "s|DT_ARCH =|DT_ARCH = $(DT_ARCH)|" \
-e "s|RC_ARCH =|RC_ARCH = $(RC_ARCH)|" \
+ -e "s|LTOCALL =|LTOCALL = $(LTOCALL)|" \
+ -e "s|LTOPREF =|LTOPREF = $(LTOPREF)|" \
-e "s|M_ARCH =|M_ARCH = $(M_ARCH)|" \
-e "s|@SYMPAT@|$(SYMPAT)|" \
-e "s|@OPENMP@|$(OPENMP)|" \
diff -ruN C:/R/ORIG/R-patched/src/gnuwin32/fixed/etc/Makeconf
C:/R/R-patched/src/gnuwin32/fixed/etc/Makeconf
--- C:/R/ORIG/R-patched/src/gnuwin32/fixed/etc/Makeconf 2015-04-19
11:02:01.000000000 -0400
+++ C:/R/R-patched/src/gnuwin32/fixed/etc/Makeconf 2015-09-08
18:09:32.898759500 -0400
@@ -14,6 +14,12 @@
DLLFLAGS+= -static-libgcc
LINKFLAGS+= -static-libgcc
+ifdef USE_LTO
+ DLLFLAGS+= -Wl,--allow-multiple-definition
+ LINKFLAGS+= -Wl,--allow-multiple-definition
+
+endif
+
## Things which are substituted by fixed/Makefile (and also -O3 -> -O2)
WIN = 32
BINPREF =
@@ -37,7 +43,7 @@
DLLTOOL = $(BINPREF)dlltool --as $(BINPREF)as $(DT_ARCH)
DLLTOOLFLAGS = -k
-NM = $(BINPREF)nm
+NM = $(BINPREF)$(LTOPREF)nm
RESCOMP = $(BINPREF)windres $(RC_ARCH)
## MAIN_LD needs to be set by the package
LINKER = $(MAIN_LD)
@@ -61,20 +67,20 @@
# was a reference to Rzlib.dll in R < 3.2.0
ZLIB_LIBS = -lz
-AR = $(BINPREF)ar
+AR = $(BINPREF)$(LTOPREF)ar
## Used by packages 'maps' and 'mapdata'
AWK = gawk
BLAS_LIBS = -L"$(R_HOME)/$(IMPDIR)" -lRblas
C_VISIBILITY =
-CC = $(BINPREF)gcc $(M_ARCH)
+CC = $(BINPREF)gcc $(M_ARCH) $(LTOCALL)
CFLAGS = -O3 -Wall $(DEBUGFLAG) -std=gnu99 @EOPTS@
CPICFLAGS =
-CPPFLAGS =
-CXX = $(BINPREF)g++ $(M_ARCH)
+CPPFLAGS =
+CXX = $(BINPREF)g++ $(M_ARCH) $(LTOCALL)
CXXCPP = $(CXX) -E
CXXFLAGS = -O2 -Wall $(DEBUGFLAG) @EOPTS@
CXXPICFLAGS =
-CXX1X = $(BINPREF)g++ $(M_ARCH)
+CXX1X = $(BINPREF)g++ $(M_ARCH) $(LTOCALL)
CXX1XFLAGS = -O2 -Wall $(DEBUGFLAG) @EOPTS@
CXX1XPICFLAGS =
CXX1XSTD = -std=c++0x
@@ -86,11 +92,11 @@
ECHO_C =
ECHO_N = -n
ECHO_T =
-FC = $(BINPREF)gfortran $(M_ARCH)
+FC = $(BINPREF)gfortran $(M_ARCH) $(LTOCALL)
FCFLAGS = -O3 $(DEBUGFLAG) @EOPTS@
# additional libs needed when linking with $(FC), e.g. on Solaris
FCLIBS =
-F77 = $(BINPREF)gfortran $(M_ARCH)
+F77 = $(BINPREF)gfortran $(M_ARCH) $(LTOCALL)
F77_VISIBILITY =
FFLAGS = -O3 $(DEBUGFLAG) @EOPTS@
FLIBS = -lgfortran
@@ -129,7 +135,7 @@
OBJC_LIBS = -lobjc
OBJCXX =
R_ARCH =
-RANLIB = $(BINPREF)ranlib
+RANLIB = $(BINPREF)$(LTOPREF)ranlib
SAFE_FFLAGS = -O3 -ffloat-store
SED = sed
## it seems some makes (but not ours) get upset if SHELL is set.
------------->8-----------------
On Tue, Sep 8, 2015 at 2:00 PM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
On 08/09/2015 12:56 PM, Avraham Adler wrote:
Hello.
As the new toolchain is going to require some changes to some of the
makefiles for Windows (such as the NM_FILTER already in place, I would
like to make some suggestions for review.
A: Link-time Optimization
I have successfully compiled R (Windows7 64bit) using link-time
optimization, but it required some monkeying around with various
files. Specifically:
1) src/gnuwin32/Mkrules.local: (no change needed)
needs '-flto -ffat-lto-objects' passed in EOPTS
2) src/gnuwin32/Mkrules: (changes needed to lines 243, 262, and 263)
AR, NM, and RANLIB all need to add 'gcc-' as a prefix to the
actual calls. For example: AR=$(BINPREF)gcc-ar
That's no good. The makefiles will need to work with both toolchains, and that doesn't look compatible with the old toolchain. I'd be happy to put in changes, subject to the following: - Existing builds should work with no change. - Users of the new toolchain should be able to try it out with a one line addition to Mkrules.local. - It would be nice to allow some flexibility, so if you get it wrong on the first try, only Mkrules.local needs to be updated, not the R-devel sources. Send me a patch that you've tested on both toolchains, and I'll put it in (after testing on the old toolchain). Duncan Murdoch
3) src/gnuwin32/Makefile: (changes needed to line 65)
DLLFLAGS needs to allow multiple definitions (for the tools
package) so the else branch needs to be changed to 'DLLFLAGS =
-Wl,--allow-multiple-definition -s'
4) src/gnuwin32/fixed/etc/Makeconf (changes needed to lines 14, 15,
40, 64, and 132)
DLLFLAGS and LINKFLAGS need '-Wl,--allow-multiple-definition'
added (lines 14 & 15)
AR, NM, and RANLIB need the 'gcc-' prefix (lines 40, 64, and 132)
Would it be possible and make sense to expose the variables AR, NM,
RANLIB, DLLFLAGS, and LINKFLAGS in Mkrules.local to allow for LTO?
B: External BLAS
I have been compiling R (Windows7 64bit) linking to an external
OpenBLAS for years now. The method requires changing
src/gnuwin32/Mkrules.local to 'USE_ATLAS = YES' and setting the proper
ATLAS_PATH (lines 30 & 31). Then changing src/extra/blas/Makefile.win
line 15 to point to the proper file. Specifically, from
'-L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lf77blas -latlas' to '
-L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lopenblas' or the
equivalent. Could the name of the BLAS library itself be made a
variable for Mkrules.local? Perhaps something like 'USE_EXTBLAS',
'EXTBLAS_PATH', and 'EXTBLASLIBS' could be used in Mkrules.local, with
EXTBLASLIBS even defaulting to '-lf77blas -latlas'. Then
src/extra/blas/Makefile.win line 15 could look something like
'-L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" $(EXTBLASLIBS)'? I admit
I have not tested this configuration yet, though.
C: Toolchain definitions
If the toolchain developed by Jeroen will be be the eventual Rtools,
as it is a multilib, src/gnuwin32/Mkrules.local lines 52?59 will
probably require a rewrite, if they are necessary at all.
Thank you,
Avi
_______________________________________________ R-SIG-windows mailing list R-SIG-windows at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-windows