How to ensure -O3 on Win64

Sat, Dec 29, 2012 4:29 AM

On 28/12/2012 00:41, Simon Urbanek wrote:

On Dec 27, 2012, at 6:08 PM, Matthew Dowle wrote:

On 27.12.2012 17:53, Simon Urbanek wrote:

On Dec 23, 2012, at 9:22 PM, Matthew Dowle wrote:

Hi,

Similar questions have come up before on the list and elsewhere but I haven't found a solution yet.

winbuilder's install.out shows data.table's .c files compiled with -O3 on Win32 but -O2 on Win64. The same happens on R-Forge. I gather that some packages don't work with -O3 so the default is -O2.

I've tried this in data.table's Makevars (entire contents) :

====
MAKEFLAGS="CFLAGS=-O3"                        # added
CFLAGS=-O3                                    # added
PKG_CFLAGS=-O3                                # added
all: $(SHLIB)                                 # no change
	mv $(SHLIB) datatable$(SHLIB_EXT)     # no change
====

but -O2 still appears in winbuilder's install.out (after -O3, and I believe the last -O is the one that counts) :

gcc -m64 -I"D:/RCompile/recent/R-2.15.2/include" -DNDEBUG     -I"d:/Rcompile/CRANpkg/extralibs215/local215/include"  -O3   -O2 -Wall -std=gnu99 -mtune=core2 -c dogroups.c -o dogroups.o

How can I ensure that data.table is compiled with -O3 on Win64?

You can't - at least not in a way that doesn't circumvent the R build
system. Also it's not portable so you don't want to mess with
optimization flags and hard-code it in your package as it's user's
choice how they setup R and its flags. You can certainly setup your R
to compile with -O3, you just can't impose that on others.

Cheers,
Simon

Thanks Simon. This makes complete sense where users compile packages on install (Unix and Mac, and I better check my settings then), but on Windows where it's more common for the user to install the pre-compiled .zip from CRAN is my concern. This came up because the new fread function in data.table wasn't showing as much of a speedup on Win64 as on Linux. I'm not 100% sure that non -O3 is the cause, but there are some function calls which get iterated a lot (e.g. isspace) and I'd seen that inlining was something -O3 did and not -O2.

In general, why wouldn't a user of a package want the best performance from -O3?

Because it doesn't work? I don't know, you said yourself that -O2 may be there since -O3 breaks - that was not the question, though. (If you are curious about that, ask on CRAN, I don't remember the answer -- note that Win64 compiler support is relatively recent).

Compilation with -O3 was not stable, including that some commonly-used 
packages segfaulted.  x64 Windows is still an under-development platform 
for gcc, and a number of issues will not be resolved until at least gcc 
4.8.0.

And BTW, -O3 is not the out-of-the-box default for any R build except 
i386 Windows, including not for the OS X binary packages.  Not sure why 
you are singling out Windows here ....

It isn't just a question of optimization level but also what optimizing 
options the compiler is built with.  We had to discard all the versions 
of gcc 4.6.x built with Graphite optimization as they produced far too 
unstable code on Windows.

Remember that even if -O3 works on your package with the current 
compiler, it may not with another one.   One reason we update the 
recommended Windows compiler so infrequently is dealing with all the 
fallout from packages which break (almost always because of previously 
unrevealed errors in their code). But, for example, we will consider if 
we want to change the recommended compiler for R 3.0.0 and it is likely 
we will change it during 2013.

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

How to ensure -O3 on Win64

Thread (6 messages)