Skip to content

src/Makevars ignored ?

9 messages · Dirk Eddelbuettel, Eric Deveaud, Kasper Daniel Hansen +1 more

Original
Eric Deveaud
# 
Hello,

I'm tring to install a Rpackage that holds some C//C++ code

as far as I understood the R library generic compilation mechanism, 
compilation of C//C++ sources is controled

1) at system level by the ocntentos RHOME/etc/Makeconf
2) at user level by the content of ~/.R/Makevars
3) at package level by the content of src/Makevars

Problem I have is that src/Makevars is ignored


see following example:

R is compiled and use the following CC and CFLAGS definition

bigmess:epactsR/src > R CMD config CC
gcc -std=gnu99
bigmess:epactsR/src > R CMD config CFLAGS
-Wall -g

so building C sources lead to the following

bigmess:epactsR/src > R CMD SHLIB index.c
gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG 
-I/usr/local/include    -fpic  -Wall -g  -c index.c -o index.o

normal, it uses defintion from RHOME/etc/Makeconf


when I set upp a ~/.R/Makevars that overwrite CC and CFLAGS definition.

bigmess:epactsR/src > cat ~/.R/Makevars
CC=gcc
CFLAGS=-O3
bigmess:epactsR/src > R CMD SHLIB index.c
gcc -I/local/gensoft2/adm/lib64/R/include -DNDEBUG  -I/usr/local/include 
    -fpic  -O3 -c index.c -o index.o
gcc -std=gnu99 -shared -L/usr/local/lib64 -o index.so index.o


OK CC and CFLAGS are honored and set accordingly to ~/.R/Makevars


but when I try to use src/Makevars, it is ignored

bigmess:epactsR/src > cat ~/.R/Makevars
cat: /home/edeveaud/.R/Makevars: No such file or directory
bigmess:epactsR/src > cat ./Makevars
CC = gcc
CFLAGS=-O3
bigmess:epactsR/src > R CMD SHLIB index.c
gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG 
-I/usr/local/include    -fpic  -Wall -g  -c index.c -o index.o


is there something I have missed, misunderstood or is there something 
wrong ?


PS I tested the ssame behaviour with various version of R from R/2.15 to 
R/3.3

     best regards

     Eric
Dirk Eddelbuettel
#
On 27 September 2016 at 09:37, Eric Deveaud wrote:
|      Hello,
| 
| I'm tring to install a Rpackage that holds some C//C++ code
| 
| as far as I understood the R library generic compilation mechanism, 
| compilation of C//C++ sources is controled
| 
| 1) at system level by the ocntentos RHOME/etc/Makeconf
| 2) at user level by the content of ~/.R/Makevars
| 3) at package level by the content of src/Makevars
| 
| Problem I have is that src/Makevars is ignored
| 
| 
| see following example:
| 
| R is compiled and use the following CC and CFLAGS definition
| 
| bigmess:epactsR/src > R CMD config CC
| gcc -std=gnu99
| bigmess:epactsR/src > R CMD config CFLAGS
| -Wall -g
| 
| so building C sources lead to the following
| 
| bigmess:epactsR/src > R CMD SHLIB index.c
| gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG 
| -I/usr/local/include    -fpic  -Wall -g  -c index.c -o index.o
| 
| normal, it uses defintion from RHOME/etc/Makeconf
| 
| 
| when I set upp a ~/.R/Makevars that overwrite CC and CFLAGS definition.
| 
| bigmess:epactsR/src > cat ~/.R/Makevars
| CC=gcc
| CFLAGS=-O3
| bigmess:epactsR/src > R CMD SHLIB index.c
| gcc -I/local/gensoft2/adm/lib64/R/include -DNDEBUG  -I/usr/local/include 
|     -fpic  -O3 -c index.c -o index.o
| gcc -std=gnu99 -shared -L/usr/local/lib64 -o index.so index.o
| 
| 
| OK CC and CFLAGS are honored and set accordingly to ~/.R/Makevars
| 
| 
| but when I try to use src/Makevars, it is ignored
| 
| bigmess:epactsR/src > cat ~/.R/Makevars
| cat: /home/edeveaud/.R/Makevars: No such file or directory
| bigmess:epactsR/src > cat ./Makevars
| CC = gcc
| CFLAGS=-O3
| bigmess:epactsR/src > R CMD SHLIB index.c
| gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG 
| -I/usr/local/include    -fpic  -Wall -g  -c index.c -o index.o
| 
| 
| is there something I have missed, misunderstood or is there something 
| wrong ?

You have not demonstrated that src/Makevars is ignored -- as it clearly
isn't, given how thousands of CRAN packages use it.

What you have done is demonstrate that you _cannot change CC and CXX_ in
src/Makevars.  And I think that was known, though maybe not as widely.

Dirk

| PS I tested the ssame behaviour with various version of R from R/2.15 to 
| R/3.3
| 
|      best regards
| 
|      Eric
| 
| ______________________________________________
| R-devel at r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel
Eric Deveaud
# 
Le 27/09/16 ? 13:31, Dirk Eddelbuettel a ?crit :
Hello Dirk,


so why ~/R/Makevars allows to change CC and not src/Makevars ?

this is pretty confusing.

in unix world mechanism of overwriting system default in user and 
package order is something standadr and consistent given the level

priority is package specific stuff overwrite user configuration that 
overwrite system default

mechanism of Makevars is NOT consistent


furthermore diggng in $RHOME/bin/config one can see the following


if test "${site}" = "yes"; then
: ${R_MAKEVARS_SITE="${R_HOME}/etc${R_ARCH}/Makevars.site"}
   if test -f "${R_MAKEVARS_SITE}"; then
     makefiles="${makefiles} -f ${R_MAKEVARS_SITE}"
   fi
fi
if test "${personal}" = "yes"; then
   if test "${R_OSTYPE}" = "windows"; then
     if test -f "${R_MAKEVARS_USER}"; then
       makefiles="${makefiles} -f ${R_MAKEVARS_USER}"
     elif test ${R_ARCH} = "/x64" -a -f "${HOME}/.R/Makevars.win64"; then
       makefiles="${makefiles} -f ${HOME}/.R/Makevars.win64"
     elif test -f "${HOME}/.R/Makevars.win"; then
       makefiles="${makefiles} -f ${HOME}/.R/Makevars.win"
     elif test -f "${HOME}/.R/Makevars"; then
       makefiles="${makefiles} -f ${HOME}/.R/Makevars"
     fi
   else
     . ${R_HOME}/etc${R_ARCH}/Renviron
     if test -f "${R_MAKEVARS_USER}"; then
       makefiles="${makefiles} -f ${R_MAKEVARS_USER}"
     elif test -f "${HOME}/.R/Makevars-${R_PLATFORM}"; then
       makefiles="${makefiles} -f ${HOME}/.R/Makevars-${R_PLATFORM}"
     elif test -f "${HOME}/.R/Makevars"; then
       makefiles="${makefiles} -f ${HOME}/.R/Makevars"
     fi
   fi
fi

it checks and honours R_MAKEVARS_USER, ~/.R/Makevars, but not src/Makevars
I'm not really familiar with the R policy about code comilation, but I 
consider this a bug but maybee I'm wrong


as a side note I solved the problem of overwritting CC for this 
particular package using MAKEFLAGS="CC=gcc" R CMD INSTALL package


     best regards

     Eric
Kasper Daniel Hansen
# 
As a package author, it is in my opinion irresponsible to override these
system settings (which is why it is also impossible).  You have no idea
what system it is being deployed on, I mean, you don't even know if the
compiler is gcc. If a user wants (say) heavy optimization they will compile
R with optimization.  (For this reason I also don't think users should
modify their ~/.R/Makevars, unless for testing purposes).  The exception is
(in my opinion) if you need to decrease the level of optimization because
you know or suspect the compiler to generate wrong code.

What you should do, is use
  PKG_CFLAGS
as documented in R-exts, 1.2.1 under "Using Makevars".

Best,
Kasper
On Tue, Sep 27, 2016 at 9:23 AM, Eric Deveaud <edeveaud at pasteur.fr> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 27 September 2016 at 15:23, Eric Deveaud wrote:

        
| so why ~/R/Makevars allows to change CC and not src/Makevars ?
| 
| this is pretty confusing.

It seems weird at first, but makes some sense when you think about it like
this:

  -- src/Makevars is inside a package and cannot / should not alter "system"
     parameters like compiler brand or version as there may only be one
     compiler, the one R was built with

     it does allow however to set compilation flags, language standards,
     include directories etc as needed to build the package

  -- (user- or system-level) Makeconf allow compiler changes, but that is
     orthogonal to the per-package config and stays local to the machine
 
| in unix world mechanism of overwriting system default in user and 
| package order is something standadr and consistent given the level
| 
| priority is package specific stuff overwrite user configuration that 
| overwrite system default
| 
| mechanism of Makevars is NOT consistent

Yes we sometimes wish we could override system Makeconf settings in a
package, but we can't.

| as a side note I solved the problem of overwritting CC for this 
| particular package using MAKEFLAGS="CC=gcc" R CMD INSTALL package

That works and is documented, but is not "portable" to other machines.

Hope this helps,  Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Eric Deveaud
    

    

      
      #
    

  


  

      
Le 27/09/16 ? 16:17, Kasper Daniel Hansen a ?crit :

            

      
      
      
    

        
as the it guy dedicated to install and maintain softs on our cluster I 
have a reasonable knowledge of the systems I work on.

I don't want to distribute any of the piece of code I was asked to made 
available on the cluster.
I just need (and succeded plus functional test succeded) to build the 
Rpackage requested by a specific software.

            

      
      
      
    

        
my question was not in a R package developer context, but in the R user 
that grabs some piece of code and is not abble to compile it because of
1) a developper that mixed C and C++ code which is legit.
2) a silly interaction beetween C and C++ symbol generation because of 
the use, in our case, of CC = gcc -std=gnu99
3) a dev that answwer: "I have no clue, in debian it works" ;-(

anyway I still convinced that if R provides a mechanisn hierachical way 
of variable overwrite pkg / user/ system it _SHOULD_ be consistent at 
all levels

my question was raised because of our install mechanism that (hopefully) 
does not allow
modification of files like ~/.R/Makevars.
I can only "play" with the sources of the software it is working on 
and//or environment variables. so I wanted  to have a temporary way of 
setting CC to be plain gcc without ISO C99 language standard support 
just for this specific R library.

            

      
      
      
    

        
in the documentation you pointed (and trust me I read it), keyword is 
_set additional_  preprocessor options and//or compiler flags

only way to _remove_ is to overwrite

back to logic.

either Makevars, whatever level, allow to overwrite CC definition
either Makevars, whatever level, disable CC redefinition

but not a mix

	Eric

  
  


  


      

    

    

      
      

        


  

        

      Eric Deveaud
    

    

      
      #
    

  


  

      
Le 27/09/16 ? 16:30, Dirk Eddelbuettel a ?crit :

            

      
      
      
    

        
yep I fully aggree with this.

            

      
      
      
    

        
I realize based on Kasper's answer and your's that I was not clear on 
the context I was working.


	Eric

  
  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
So now we have some important context for your original question.

I understand why you want "symmetry" but because of the reasons Dirk
outlines, I personally think it is a bad idea, ie. I disagree with your
statement "anyway I still convinced that if R provides a mechanisn
hierachical way of variable overwrite pkg / user/ system it _SHOULD_ be
consistent at all levels".

The mechanism for achieving what you want - overriding CC on a local
machine -  is using a ~/.R/Makevars file.  I am not sure that this can be
done in a package specific manner (I don;t do this very often), but you
could create this file ('this file" being ~/.R/Makevars), install the
package and then remove it.  Remember, this is an install-time setting, so
you don't need the users of your system to be involved in this. That way
you could test your proposed fix.  If the fix works, it seems to me like it
should be included in the package source by the package maintainer, perhaps
using a configure script, but that is ultimately something which is up to
the package maintainer.

Best,
Kasper

        
On Tue, Sep 27, 2016 at 10:52 AM, Eric Deveaud <edeveaud at pasteur.fr> wrote:

        

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Bob Rudis
    

    

      
      #
    

  


  

      
You're then asking CRAN to violate your "ideal contract" w/r/t compiler
switching inside src/Makevars since CRAN needs to setup and produce
standard, predictable, repeatable builds, including binary generation for
two platforms (much to Dirk's chagrin, there _are_ other operating systems
besides Debian linux ;-)

I wouldn't want CRAN to honor said compiler switching inside src/Makevars
but the way you're describing the perceived contract, you're implying they
should.

I totally understand your posit, but there's a larger universe to consider
than internal package builds. In a sense you really want an R version of
Rule of Acquisition #17 "A contract is a contract is a contract... but only
on my internal systems." and that doesn't seem like a good idea to me when
the larger universe of CRAN and the need for baseline standard package
builds are considered.



On Tue, Sep 27, 2016 at 11:19 AM, Kasper Daniel Hansen <

        
kasperdanielhansen at gmail.com> wrote: