Skip to content

any suggestions to deal with 'Argument list too long' for a R CMD check?

8 messages · Jeff Ryan, Mathieu Ribatet, Whit Armstrong +1 more

Original
Whit Armstrong
# 
Since, gcc was using upwards of 2gb of ram to compile my package, I
just split all the functions into individual files.

I guess I'm too clever for myself, because now I get hit with the
"Argument list too long" error.

Is there a way to deal with this aside from writing my own configure
script (which could possibly feed the gcc commands one by one).

-Whit

RHEL 5
[whit at linuxsvr]$ uname -a
Linux linuxsvr.kls.corp 2.6.18-92.1.18.el5 #1 SMP Wed Nov 5 09:00:19
EST 2008 x86_64 x86_64 x86_64 GNU/Linux

from 00install.out:

* Installing *source* package 'RFincad' ...
** libs
g++ -I/usr/local/lib64/R/include  -I/usr/local/include
-I./RAbstraction -I./R.tslib.backend -I./tslib -fpic  -g -O2 -c
aaAccrual_days_act252.cpp -o aaAccrual_days_act2\
52.o
make: execvp: g++: Argument list too long
make: *** [aaAccrual_days_act252.o] Error 127
ERROR: compilation failed for package 'RFincad'
** Removing '/home/whit/dvl/fincad.utils/fincad.generate.R/RFincad.Rcheck/RFincad'
Jeff Ryan
# 
Never used, but I seem to recall that this was good for that:

http://en.wikipedia.org/wiki/Xargs

Jeff
On Tue, Dec 9, 2008 at 3:13 PM, Whit Armstrong <armstrong.whit at gmail.com> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Whit Armstrong
    

    

      
      #
    

  


  

      
I love xargs for all kinds of stuff, but I really don't want to write
my own configure script.

looks like simple solution is to recombine into one big source file
rather than splitting each function out.  pita...

-Whit

        
On Tue, Dec 9, 2008 at 4:23 PM, Jeff Ryan <jeff.a.ryan at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Mathieu Ribatet
    

    

      
      #
    

  


  

      
Just a suggestion. I googled for your error message you got e.g. "make 
... error 127" and I got

Yeah, error 127 indicates that make can't find a program that it needs to
run a command.  My guess would be that she doesn't have g++ installed
(some distributions package it separately from gcc so it is possible to
have the c compiler installed but not the c++ compiler)



Maybe it will be worthwhile to check if you have installed every 
required header files or needed compilers/libraries.
But as I mentioned, this is just a suggestion.
Cheers,
Mathieu

Whit Armstrong a ?crit :

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Whit Armstrong
    

    

      
      #
    

  


  

      
Thanks for the suggestion.  this package was building successfully
until I split it up into smaller files.

(there are 4489 files in the src directory)

I suspect it's a hard limit in the linux kernel.

http://www.linuxjournal.com/article/6060

since R CMD check is generating the actual commands that are passed to
bash, it's a little out of my control (unless I write my own config
script, which I do not want to do).

-Whit

        
On Tue, Dec 9, 2008 at 5:07 PM, Mathieu Ribatet <mathieu.ribatet at epfl.ch> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Tue, 9 Dec 2008, Whit Armstrong wrote:

        

            

      
      
      
    

        
Well 'R CMD check' is not doing so: perhaps SHLIB called by INSTALL is.

What I suspect is happening is that there is a link command starting with 
'g++' that is too long.

POSIX requires at least 4096 arguments (which you will exceed), but Linux 
2.6 seems to have a limit of much more (unlimited in the current kernel, 
which yours is not, 32767 in 2.16.14).  It is more likely that you are 
hititng the length limit for the command line, and one way out is to use 
short file names.

One useful ref: http://www.in-ulm.de/~mascheck/various/argmax

Note that having all the functions in one file can help the optimizer, but 
there is another possibility, to have several files of closely related 
functions.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Whit Armstrong
    

    

      
      #
    

  


  

      
Thanks for the link, Prof. Ripley.

            

      
      
      
    

        
interesting...

on my system:

[whit at linuxsvr src]$ getconf ARG_MAX
131072
[whit at linuxsvr src]$

Let me try on another machine w/ a more recent kernel before I take up
more of everyone's time.

-Whit

  
  


  


      

    

    

      
      

        


  

        

      Whit Armstrong
    

    

      
      #
    

  


  

      
This package built successfully on ubuntu 8.10.

Thanks to everyone for the suggestions!

Since the new method (splitting the files) will fail for anyone not
using a very recent linux kernel, I think I should revert to the old
consolidated files.

I suspect there is little interest from R-core in changing the build
script to link the files incrementally (is that possible).  To my
knowledge no other packages out there have such an insane amount of
files.

-Whit

        
On Tue, Dec 9, 2008 at 5:59 PM, Whit Armstrong <armstrong.whit at gmail.com> wrote: