Skip to content

lme4 + R 2.11.0 + mac unavailable

12 messages · Martin Maechler, Ken Beath, John Maindonald +2 more

Original
Martin Maechler
# 
Dear Peter,
note that I am CC'ing this reply to the "R - Mac" mailing list, on
which there are knowledgable people for helping you with some of the
Mac specific problems
(more inline below).
On Sun, Jul 25, 2010 at 20:36, Peter Goff <peter.t.goff at vanderbilt.edu> wrote:
Ok, I assume that's close enough to Vanderbilt ..
See, that's a clue:  'gcc' is the GNU C compiler and that (and more)
are absolutely musts for installing many R packages from source.

The readers of R-SIG-Mac will be able to give more exact details,
but in order to get a version of gcc (and other tools) working nicely
with your version of R,
you'll need to install (parts of) the "Xcode" development tools that
Apple provides to you (for free, but you need to "get them").

Best regards,
Martin
Ken Beath
#
On 27/07/2010, at 9:57 PM, Martin Maechler wrote:

            

      
      
      
    

        
Not that difficult, it is on a DVD that is included with the Mac (I think Extra Installs or similar), or alternatively try http://developer.apple.com/mac/, register and then download.

Ken

  
  


  


      

    

    

      
      

        


  

        

      John Maindonald
    

    

      
      #
    

  


  

      
Binaries for lme4 are not for the time being available either on
CRAN or on r-forge?  

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm

        
On 27/07/2010, at 9:57 PM, Martin Maechler wrote:

        

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Ken Beath
    

    

      
      #
    

  


  

      
On Tue, July 27, 2010 11:07 pm, John Maindonald wrote:

            

      
      
      
    

        
There was a discussion about this on R-Sig-ME. It fails one of the checks
on the machine used to build the mac packages. It may pass on machines
with a later version of MacOS, but there didn't seem to be a consensus of
whether failing the checks was a problem.

Ken

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      Martin Maechler
    

    

      
      #
    

  


  

          

      
      
      
    

        

        
KB> On Tue, July 27, 2010 11:07 pm, John Maindonald wrote:

        
>> Binaries for lme4 are not for the time being available either on
    >> CRAN or on r-forge?
    >> 

    KB> There was a discussion about this on R-Sig-ME. It fails one of the checks
    KB> on the machine used to build the mac packages. It may pass on machines
    KB> with a later version of MacOS, but there didn't seem to be a consensus of
    KB> whether failing the checks was a problem.

The underlyign issue seems to remain unresolved.
My best guess is still to low-level bugs somewhere in the libraries
used.

However, to help users,
I'm willing to *not* run those tests when *running* on the Mac.

What does  
     Sys.info()[["sysname"]]

return on the Mac, i.e., different versions of R running on the
Mac?

            

      
      
      
    

        
Is that true?

Regards,
Martin Maechler



    >> John Maindonald             email: john.maindonald at anu.edu.au
    >> phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
    >> Centre for Mathematics & Its Applications, Room 1194,
    >> John Dedman Mathematical Sciences Building (Building 27)
    >> Australian National University, Canberra ACT 0200.
    >> http://www.maths.anu.edu.au/~johnm
    >>

        
>> On 27/07/2010, at 9:57 PM, Martin Maechler wrote:

        
>> 
    >>> Dear Peter,
    >>> note that I am CC'ing this reply to the "R - Mac" mailing list, on
    >>> which there are knowledgable people for helping you with some of the
    >>> Mac specific problems
    >>> (more inline below).
    >>> 
    >>> On Sun, Jul 25, 2010 at 20:36, Peter Goff <peter.t.goff at vanderbilt.edu>

        
>>> wrote:

        
>>>> Martin,
    >>>> 
    >>>> I saw your series of posts on this topic, but sadly I am still unable
    >>>> to
    >>>> install lme4.
    >>>> 
    >>>> I am in the nascent stages of learning R and, at this point, I cannot
    >>>> even
    >>>> load the program "arm" that is key to my program of study (I'm working
    >>>> with
    >>>> Gelman & Hill's Data Analysis using Regression and
    >>>> Multilevel/Hierarchical
    >>>> Models to get acclimated to R).
    >>>> 
    >>>> Here's my quandary thus far:
    >>>> 
    >>>> (1) I download and install R for Macs (version 2.11.1)
    >>>> (2) I try to find lme4 through the "Package Installer" only to find
    >>>> that it
    >>>> does not exist.
    >>>> (3) I find your string of postings on the web and try to install using:
    >>>> install.packages("lme4", type = "source")
    >>>> and I get the following error message:
    >>>> 
    >>>> --- Please select a CRAN mirror for use in this session --- (I chose
    >>>> USA,
    >>>> MI)
    >>> 
    >>> Ok, I assume that's close enough to Vanderbilt ..
    >>> 
    >>>> trying URL 'http://cran.mtu.edu/src/contrib/lme4_0.999375-34.tar.gz'
    >>>> Content type 'application/x-gzip' length 1028012 bytes (1003 Kb)
    >>>> opened URL
    >>>> ==================================================
    >>>> downloaded 1003 Kb
    >>>> 
    >>>> * installing *source* package ?lme4? ...
    >>>> ** libs
    >>>> *** arch - i386
    >>>> gcc -arch i386 -std=gnu99
    >>>> -I/Library/Frameworks/R.framework/Resources/include
    >>>> -I/Library/Frameworks/R.framework/Resources/include/i386
    >>>> -I/usr/local/include
    >>>> -I"/Library/Frameworks/R.framework/Versions/2.11/Resources/library/Matrix/include"
    >>>> -I"/Library/Frameworks/R.framework/Resources/library/stats/include"
    >>>> -fPIC
    >>>> -g -O2 -c init.c -o init.o
    >>>> /bin/sh: gcc: command not found
    >>> 
    >>> See, that's a clue:  'gcc' is the GNU C compiler and that (and more)
    >>> are absolutely musts for installing many R packages from source.
    >>> 
    >>> The readers of R-SIG-Mac will be able to give more exact details,
    >>> but in order to get a version of gcc (and other tools) working nicely
    >>> with your version of R,
    >>> you'll need to install (parts of) the "Xcode" development tools that
    >>> Apple provides to you (for free, but you need to "get them").
    >>> 
    >>> Best regards,
    >>> Martin
    >>> 
    >>>> make: *** [init.o] Error 127
    >>>> ERROR: compilation failed for package ?lme4?
    >>>> * removing
    >>>> ?/Library/Frameworks/R.framework/Versions/2.11/Resources/library/lme4?
    >>>> 
    >>>> The downloaded packages are in
    >>>> 
    >>>> ?/private/var/folders/ty/tyL1SdtKGlObzhxQbBUTek+++TI/-Tmp-/Rtmpt4fli2/downloaded_packages?
    >>>> Updating HTML index of packages in '.Library'
    >>>> Warning message:
    >>>> In install.packages("lme4", type = "source") :
    >>>> installation of package 'lme4' had non-zero exit status
    >>>> 
    >>>> 
    >>>> 
    >>>> Days and hours later I am at a loss as to how I should proceed. Any
    >>>> advice
    >>>> would be greatly appreciated!
    >>>> 
    >>>> Thanks,
    >>>> ~Peter
    >>>> 
    >>>> Peter Trabert Goff
    >>>> PhD student
    >>>> Department of Leadership, Policy, and Organizations
    >>>> Vanderbilt University
    >>> 
    >>> _______________________________________________
    >>> R-SIG-Mac mailing list
    >>> R-SIG-Mac at stat.math.ethz.ch
    >>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
    >> 
    >> _______________________________________________
    >> R-SIG-Mac mailing list
    >> R-SIG-Mac at stat.math.ethz.ch
    >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
    >>

  
  


  


      

    

    

      
      

        


  

        

      John Maindonald
    

    

      
      #
    

  


  

      
It gives "Darwin" on my machine.

This is an issue (binaries unavailable) of a type that is likely to
recur from time to time, and not just for OSX implementations.  
At the very least, ready access to reasonably complete information 
on the status of packages, both on r-forge and on CRAN and
brought together in one place, is surely desirable for widely
used packages.   The packages check summary will seem a
forbidding place fro relative novices.

Thanks to those who are cogitating about how to handle such
issues, and trying to smooth the way ahead.

There might be a link on the relevant CRAN binaries page:
"The binary for the package that you want is not available.  Click 
here for details of the current status, and for a note on your options
for gaining access to it."

While compiling from source is a suitable recourse for those
who are accustomed to compile packages, encouraging
novice to compile from source seems in general a bad idea.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm

        
On 30/07/2010, at 9:51 AM, Martin Maechler wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Simon Urbanek
    

    

      
      #
    

  


  

      
On Jul 29, 2010, at 7:51 PM, Martin Maechler wrote:

        

            

      
      
      
    

        
... why don't you just compare the values with tolerance? That seems to make more sense that to disable the test altogether ...

Cheers,
Simon

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Martin Maechler
    

    

      
      #
    

  


  

      
On Fri, Jul 30, 2010 at 17:29, Simon Urbanek

        
<simon.urbanek at r-project.org> wrote:

            

      
      
      
    

        
Doug and I have explained in several occasions (but maybe not on R-SIG-Mac)
that this is *NOT* the point here.
It's not about "basically the same computations" it's about
identical computations, i.e. from identical numerical matrices and so
the use of identical() has been very much on purpose, and is different
from the well justified typical use of
all.equal() which we use in many other places.

And what people where reporting *is* a bit disturbing: The same
computations giving different answers for *exactly* the same
computation, e.g., depending if they had used
library(lme4)  or  require(lme4)  to load the package ...
and the difference was not just the last significant bit, but changes
at about the 7-th significant digit.

Martin

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      John Maindonald
    

    

      
      #
    

  


  

      
Is optimisation of code a possibility, i.e., the same algebraic calculations 
done  in a slightly different way depending on the state of various registers?
E.g., (ab)c vs a(bc), but it would almost certainly be more subtle than that.  
That seems to me the sort of thing that is likely to be Mac-specific.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm

        
On 31/07/2010, at 5:09 AM, Martin Maechler wrote:

        

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
Let us be clear about what the issue seems to be.  On i386 Mac (and 
not on x86_64 Mac)

library(lme4)
y <- (1:20)*pi; x <- (1:20)^2;group <- gl(2,10)
M2. <- lmer (y ~ 1 + x + (1 + x | group))
M2 <- lmer (y ~ x + ( x | group))
identical(fixef(M2), fixef(M2.))

usually fails the first time but if you run it again it is usually 
true, e.g. following on gave

            

      
      
      
    

        
(Intercept)           x
  15.2354383   0.1855865
(Intercept)           x
  15.2354486   0.1855864

            

      
      
      
    

        
(Intercept)           x
  15.2354383   0.1855865

I think 'later version of MacOS' is most likely someone not 
understanding that x86_64 is the default for 'R' on Snow Leopard. 
I've just checked this on Leopard and Snow Leopard.  *But* it is not 
repeatable, as sometimes I get TRUE the first run in a session -- and 
once that happens it seems to keep happening until a reboot or I 
reinstall lme4.

I ran this under valgrind (on Leopard: there is now experimental 
valgrind support for SL, but I've not installed it as yet) and saw no 
reports of problems.  However, on one of the times I tried it under 
valgrind  the result was further away:

            

      
      
      
    

        
(Intercept)           x
  15.2381811   0.1856179
and persistent.

For completeness, I also saw this using R's reference BLAS rather than 
the (default on a CRAN build) Apple 'veclib' BLAS.

So the issue appears to be one of repeating the calculation and 
getting different answers.  That does look like a i386-Mac-specific 
bug, but whether this is worth not distributing a binary for is 
something for the lme4 authors to judge -- if not then a 
platform-specific test opt-out seems the best way forward.

        
On Sat, 31 Jul 2010, John Maindonald wrote:

        

            

      
      
      
    

        
But the Mac shares a compiler with lots of other platforms, and the 
gcc optimizer for i386 is pretty much shared across platforms (albeit 
the Mac gcc is now rather old, but is of similar vintage to that used 
on Windows for R 2.11.x).  In any case, this is not (in my 
experiments) an issue of numerical accuracy but of repeatability,

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Maechler
    

    

      
      #
    

  


  

      
On Sat, Jul 31, 2010 at 04:25, John Maindonald

        
<john.maindonald at anu.edu.au> wrote:

            

      
      
      
    

        
I'm really not a Mac specialist and neither one in those low level
system / CPU computations,  but indeed, exactly something like that
has been my current guess about what's happening...

            

      
      
      
    

        
yes, more subtle ( I hope).  We know that computer arithmetic does
*not*  fulfill most of these basic arithmetic laws, but remember that
here, we cannot imagine how the numeric matrices that are used in the
computations could *start*  differently, and then I'd wonder how
differences came about...
and recall that the differences we've seen where not just in the last
few significant bits

            

      
      
      
    

        
interesting...  so you have seen other evidence of such behavior?

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Martin Maechler
    

    

      
      #
    

  


  

      
On Sat, Jul 31, 2010 at 13:24, Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote:

            

      
      
      
    

        
Thank you very much, Brian, for your experiments and concluding thoughts.
At the moment (since about yesterday), the R-forge version of lme4 has a test

  if(Sys.info()[["sysname"]] != "Darwin")

before two of the stopifnot(identical(..), identical(..))  checks.
Now if I understood correctly, I could constrain the if(..) close even
further by
requiring both "Darwin" and "i386"  for "not checking".
So, I'd use

    Sys.info()[c("sysname","machine")] == c("Darwin", "i386")

or would you recommend something better?

Thanks again,
Martin