Hello R Community,
I have recently started using R for my PhD work. Aside of that I am Mac and iOS developer since few years. As far I can see R package for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
Correctly if I am wrong, but the only reason of such separation (rather doing single universal version for all PPC, i386 and x86_64) is Carbon dependency for various R packages? So 32-bit version is the one that uses native Quartz or OpenGL rendering, while 64-bit version is the one that can drive over 2GB of memory but falls back to X11 as Carbon isn't available for 64-bit.
Can anyone tell me what are the packages that still relay on Carbon (quartz renderer, rgl? anything else) that would need to be ported to Cocoa so 64-bit version will offer full functionality as 32-bit counterparts on Mac, instead of falling back to X11? I guess I could help quickly porting these parts into Cocoa, so we got single (3 architecture) binary.
Cheers,
Adam Strzelecki | nanoant.com
BTW. Who's the Mac package maintainer in R project?
On Dec 7, 2011, at 9:07 AM, Adam Strzelecki wrote:
Hello R Community,
I have recently started using R for my PhD work. Aside of that I am
Mac and iOS developer since few years. As far I can see R package
for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
Most of them build as universal unless you do the make/compile stuff
on your own. The compile object code gets put in the .../pckgname/
libs/ directory and mine have x86_64, ppc, and i386 subdirectories.
http://r.research.att.com/
Correct [me] if I am wrong, but the only reason of such separation
(rather doing single universal version for all PPC, i386 and x86_64)
is Carbon dependency for various R packages? So 32-bit version is
the one that uses native Quartz or OpenGL rendering, while 64-bit
version is the one that can drive over 2GB of memory but falls back
to X11 as Carbon isn't available for 64-bit.
Can anyone tell me what are the packages that still relay on Carbon
(quartz renderer, rgl? anything else) that would need to be ported
to Cocoa so 64-bit version will offer full functionality as 32-bit
counterparts on Mac, instead of falling back to X11? I guess I could
help quickly porting these parts into Cocoa, so we got single (3
architecture) binary.
I am running 64 bit Mac almost exclusively with the Mac GUI, and X11
is only loaded for a few packages that rely on its windowing
(typically ones that rely on TcL/Tk). I believe the 32 bit version
addresses whatever portion of 4 GB is not occupied by system or other
programs (not 2 GB) and would be faster and more memory efficient if
were used on a machine with memory limited to 4 GB. There are two
GUIs, one 32 bit and one 64 bit and they both use Quartz. I'm
reasonably sure the Tcl/Tk dependency on X11 is cross-architecture.
Cheers,
--
Adam Strzelecki | nanoant.com
BTW. Who's the Mac package maintainer in R project?
Simon Urbanek.
He reads this list regularly and will be by to correct my errors.
On Dec 7, 2011, at 9:07 AM, Adam Strzelecki wrote:
Hello R Community,
I have recently started using R for my PhD work. Aside of that I am Mac and iOS developer since few years. As far I can see R package for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
Not really - it's one R with three architectures (i386, x86_64 and ppc) - you can choose them using R --arch <arch>.
Correctly if I am wrong, but the only reason of such separation (rather doing single universal version for all PPC, i386 and x86_64) is Carbon dependency for various R packages?
Yes, you are wrong, that is not the rationale. The main reason is the same as why Apple uses separate files for i386, x86_64 and ppc: portability (see gcc, for example). In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context. Think about the consequences - R code can be conditioned on the architecture, so when building packages, you have to run them for every architecture separately which is unnecessary difficult with universal binaries (you can't simply use -arch foo -arch bar like with gcc). It's much easier if you use the classic multi-lib approach where you can run separate architectures at will. (Technically the only difference is that we use thin binaries instead of fat binaries, so it's only the format on disk, not anything else)
As a side note - the R framework and all its libraries are universal as you can see, so "native" Mac apps that link the framework can be universal (and e.g. the GUI is).
Also your subject doesn't make sense - all architectures in R have exactly the same functionality. None of them is "limited" in any way. A side-effect of the multi-lib approach is that users can choose to compile only specific architectures (more for convenience than anything), but CRAN only builds complete packages (i.e., all architectures).
Cheers,
Simon
So 32-bit version is the one that uses native Quartz or OpenGL rendering, while 64-bit version is the one that can drive over 2GB of memory but falls back to X11 as Carbon isn't available for 64-bit.
Can anyone tell me what are the packages that still relay on Carbon (quartz renderer, rgl? anything else) that would need to be ported to Cocoa so 64-bit version will offer full functionality as 32-bit counterparts on Mac, instead of falling back to X11? I guess I could help quickly porting these parts into Cocoa, so we got single (3 architecture) binary.
Cheers,
--
Adam Strzelecki | nanoant.com
BTW. Who's the Mac package maintainer in R project?
Another important thing to note is that the multi-lib approach works well
on other platforms like Windows and Linux where "universal binaries" do not
exist. Using the same approach on OS X makes for one less special case to
worry about.
-Charlie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20111210/b2d54aff/attachment.html>
I have recently started using R for my PhD work. Aside of that I am Mac and iOS developer since few years. As far I can see R package for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
Not really - it's one R with three architectures (i386, x86_64 and ppc) - you can choose them using R --arch <arch>.
I've used work "flavor" intentionally, I know 32-bit version is FAT binary for both i386 and PPC (which are both 32-bit FYI) :)
Anyway I refer only to R.app and R64.app asking for the reason of coming in "two flavors". It is very uncommon and against OSX standards to provide two separate apps for 64-bits and 32-bits, where one can choose that via Finder (but usually it is not necessary at all), see: http://kb2.adobe.com/cps/509/cpsid_50983/images/Safari_32-bit_mode1.png
Correctly if I am wrong, but the only reason of such separation (rather doing single universal version for all PPC, i386 and x86_64) is Carbon dependency for various R packages?
Yes, you are wrong, that is not the rationale. The main reason is the same as why Apple uses separate files for i386, x86_64 and ppc: portability (see gcc, for example).
Frankly I do not see any place where Apple does use separate files for each supported architecture, here's output from `file` on couple of Apple apps and libs on my Lion 10.7.2, all of them are FAT (universal binaries):
$ file /usr/lib/libgcc_s.1.dylib
/usr/lib/libgcc_s.1.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libgcc_s.1.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libgcc_s.1.dylib (for architecture i386): Mach-O dynamically linked shared library i386
$ file /usr/bin/gcc
/usr/bin/gcc: Mach-O universal binary with 2 architectures
/usr/bin/gcc (for architecture i386): Mach-O executable i386
/usr/bin/gcc (for architecture x86_64): Mach-O 64-bit executable x86_64
$ file /Applications/Safari.app/Contents/MacOS/Safari
/Applications/Safari.app/Contents/MacOS/Safari: Mach-O universal binary with 2 architectures
/Applications/Safari.app/Contents/MacOS/Safari (for architecture i386): Mach-O executable i386
/Applications/Safari.app/Contents/MacOS/Safari (for architecture x86_64): Mach-O 64-bit executable x86_64
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture. So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
Think about the consequences - R code can be conditioned on the architecture, so when building packages, you have to run them for every architecture separately which is unnecessary difficult with universal binaries (you can't simply use -arch foo -arch bar like with gcc). It's much easier if you use the classic multi-lib approach where you can run separate architectures at will. (Technically the only difference is that we use thin binaries instead of fat binaries, so it's only the format on disk, not anything else)
Using multiple -arch arguments is not the only way of producing universal binary, you may have classic separate build process for each architecture, then during post-build you may use "lipo" command to produce single FAT binary. Anyway using separate build process for each architecture on Mac is only justified if each arch has different library dependencies and #ifdef/#ifs are not enough.
I have nothing against of having separate libraries if the user does not have to care about it. So having two R.app and R64.app IMHO only leads to confusion.
As a side note - the R framework and all its libraries are universal as you can see, so "native" Mac apps that link the framework can be universal (and e.g. the GUI is).
Yes, I know /Library/Frameworks/R.framework/R is single universal library, but I do not understand why /Applications/R.app isn't :)
Also your subject doesn't make sense - all architectures in R have exactly the same functionality. None of them is "limited" in any way. A side-effect of the multi-lib approach is that users can choose to compile only specific architectures (more for convenience than anything), but CRAN only builds complete packages (i.e., all architectures).
If they both have same functionality then I don't see any benefit of using 32-bit version on 64-bit OS and CPU, since 64-bit code is faster, can address more than 4GB RAM so load bigger datasets. 2GB limit refers to Windows, so was wrong with that in former mail.
However in case of R for OSX, 32-bit rgl package uses native OpenGL rendering via Carbon+AGL API, while 64-bit version uses X11 - Carbon is deprecated 32-bit only API. So this makes difference.
That's why I was asking whether there are any other packages or components of R that relay on some deprecated 32-bit libs such as Carbon and offered my help to port these parts to 64-bit Cocoa API bringing native interface.
Charlie wrote:
Another important thing to note is that the multi-lib approach works well on other platforms like Windows and Linux where "universal binaries" do not exist. Using the same approach on OS X makes for one less special case to worry about.
If everybody was thinking like that we would all use Windows and drive only Ford cars, because it is simpler when everybody uses the same stuff :) For me FAT binaries is an advantage not a problem. Windows is just a mess, so I wouldn't treat Windows as a reference here. Linux makes you do not care about architecture once you install it (ie. Ubuntu or other distro) and use built in package manager. You just select package "R" and it just copies appropriate architecture binaries, and you get single R application.
Once again, the reason of raising this discussion was a purpose of having two separate apps - R64.app and R.app as it is very uncommon for Mac users and IMHO leads to confusion. I cannot name single other application that makes such distinction. I believe there should be only one R.app using FAT binary for all archs, running best architecture (like 64-bit on recent Macs) where possible. Also all packages referring to Carbon API should be rewritten for Cocoa.
Altogether please forgive me if I may sound like "mr wise guy" here. My only intention is to help making R more Mac user friendly.
Cheers,
On 14 Dec 2011, at 4:24 PM, Adam Strzelecki wrote:
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture. So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
I?ll second that: It makes AppleScripting easier too: Calling ?R.app? always works, instead of every user having to customise scripts with their app version R64.app vs R.app
I work around this by deleting one binary and renaming R64 to R.app
Cheers,
tim
On Dec 14, 2011, at 11:24 AM, Adam Strzelecki wrote:
Simon wrote:
I have recently started using R for my PhD work. Aside of that I am Mac and iOS developer since few years. As far I can see R package for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
Not really - it's one R with three architectures (i386, x86_64 and ppc) - you can choose them using R --arch <arch>.
I've used work "flavor" intentionally, I know 32-bit version is FAT binary for both i386 and PPC (which are both 32-bit FYI) :)
Anyway I refer only to R.app and R64.app asking for the reason of coming in "two flavors". It is very uncommon and against OSX standards to provide two separate apps for 64-bits and 32-bits, where one can choose that via Finder (but usually it is not necessary at all), see: http://kb2.adobe.com/cps/509/cpsid_50983/images/Safari_32-bit_mode1.png
Correctly if I am wrong, but the only reason of such separation (rather doing single universal version for all PPC, i386 and x86_64) is Carbon dependency for various R packages?
Yes, you are wrong, that is not the rationale. The main reason is the same as why Apple uses separate files for i386, x86_64 and ppc: portability (see gcc, for example).
Frankly I do not see any place where Apple does use separate files for each supported architecture,
Try ls /usr/lib/gcc/*/4.2.1 and notice separate directories for ppc vs intel (first level) and 32-bit vs 64-bit (second level; top dir is 32-bit 64-has its own subdir).
ginaz:bin$ ls /usr/lib/gcc/*/4.2.1
/usr/lib/gcc/i686-apple-darwin10/4.2.1:
crt3.o install-tools libgcc_eh.a libgfortran.a libgomp.a x86_64
finclude libcc_kext.a libgcc_static.a libgfortranbegin.a libgomp.spec
include libgcc.a libgcov.a libgfortranbegin.la libstdc++.dylib
/usr/lib/gcc/powerpc-apple-darwin10/4.2.1:
crt2.o include libgcc.a libgcov.a libgfortranbegin.la libstdc++.dylib
crt3.o install-tools libgcc_eh.a libgfortran.a libgomp.a ppc64
finclude libcc_kext.a libgcc_static.a libgfortranbegin.a libgomp.spec
here's output from `file` on couple of Apple apps and libs on my Lion 10.7.2, all of them are FAT (universal binaries):
$ file /usr/lib/libgcc_s.1.dylib
/usr/lib/libgcc_s.1.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libgcc_s.1.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libgcc_s.1.dylib (for architecture i386): Mach-O dynamically linked shared library i386
$ file /usr/bin/gcc
/usr/bin/gcc: Mach-O universal binary with 2 architectures
/usr/bin/gcc (for architecture i386): Mach-O executable i386
/usr/bin/gcc (for architecture x86_64): Mach-O 64-bit executable x86_64
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture.
That's why R framework does that so the discussion is really moot.
So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
If they want to do that. However, with R users typically don't want to do that - you are missing important points - see below. There is nothing preventing you from lipo'ing R from R64.app into R.app if that's what you want. It doesn't change anything, though - it's still exactly the same binary.
Think about the consequences - R code can be conditioned on the architecture, so when building packages, you have to run them for every architecture separately which is unnecessary difficult with universal binaries (you can't simply use -arch foo -arch bar like with gcc). It's much easier if you use the classic multi-lib approach where you can run separate architectures at will. (Technically the only difference is that we use thin binaries instead of fat binaries, so it's only the format on disk, not anything else)
Using multiple -arch arguments is not the only way of producing universal binary, you may have classic separate build process for each architecture, then during post-build you may use "lipo" command to produce single FAT binary. Anyway using separate build process for each architecture on Mac is only justified if each arch has different library dependencies and #ifdef/#ifs are not enough.
Please think for a moment and re-read what I said. This is not about compilation, but about R. You are completely missing the fact that there is interpreted code involved that can be (and in practice is) arch-conditional. But, again, I don't think you wanted to go there - this is about the package build process which is a whole different story and one that's certainly not Mac-specific.
I have nothing against of having separate libraries if the user does not have to care about it. So having two R.app and R64.app IMHO only leads to confusion.
As a side note - the R framework and all its libraries are universal as you can see, so "native" Mac apps that link the framework can be universal (and e.g. the GUI is).
Yes, I know /Library/Frameworks/R.framework/R is single universal library, but I do not understand why /Applications/R.app isn't :)
a) historical reason (64-bit R was optional) b) easier way to choose architecture c) capability to run them in parallel
Also your subject doesn't make sense - all architectures in R have exactly the same functionality. None of them is "limited" in any way. A side-effect of the multi-lib approach is that users can choose to compile only specific architectures (more for convenience than anything), but CRAN only builds complete packages (i.e., all architectures).
If they both have same functionality then I don't see any benefit of using 32-bit version on 64-bit OS and CPU, since 64-bit code is faster, can address more than 4GB RAM so load bigger datasets.
That is plain wrong. 64-bit is not faster, it can be both slower or faster depending on the task. And it always uses more memory. There are plenty of machines with 4GB of RAM or less where running 64-bit R can cause more trouble than you want.
2GB limit refers to Windows, so was wrong with that in former mail.
However in case of R for OSX, 32-bit rgl package uses native OpenGL rendering via Carbon+AGL API, while 64-bit version uses X11 - Carbon is deprecated 32-bit only API. So this makes difference.
That's why I was asking whether there are any other packages or components of R that relay on some deprecated 32-bit libs such as Carbon and offered my help to port these parts to 64-bit Cocoa API bringing native interface.
Fell free, this has nothing to do with R, though. You may want to ask the maintainers of the packages in question whether they would welcome a patch.
Charlie wrote:
Another important thing to note is that the multi-lib approach works well on other platforms like Windows and Linux where "universal binaries" do not exist. Using the same approach on OS X makes for one less special case to worry about.
If everybody was thinking like that we would all use Windows and drive only Ford cars, because it is simpler when everybody uses the same stuff :) For me FAT binaries is an advantage not a problem. Windows is just a mess, so I wouldn't treat Windows as a reference here. Linux makes you do not care about architecture once you install it (ie. Ubuntu or other distro) and use built in package manager. You just select package "R" and it just copies appropriate architecture binaries, and you get single R application.
That's wrong as well, you can run multi-lib on Linux (typically it's done on x64 systems to run legacy x86 code, but it is very common for other architectures like MIPS). You choose the architecture the same way as on OS X using R --arch <arch>.
Once again, the reason of raising this discussion was a purpose of having two separate apps - R64.app and R.app as it is very uncommon for Mac users and IMHO leads to confusion. I cannot name single other application that makes such distinction. I believe there should be only one R.app using FAT binary for all archs, running best architecture (like 64-bit on recent Macs) where possible. Also all packages referring to Carbon API should be rewritten for Cocoa.
Altogether please forgive me if I may sound like "mr wise guy" here. My only intention is to help making R more Mac user friendly.
Well, IMHO you don't make much sense - the only thing you seem to object is the fact that we have R.app/R64.app instead of fat R.app. However, that makes no difference whatsoever - it is just a convention that comes from the way R is run (the GUI is just an add-on, it's not "the R") and either approach works just as well. It doesn't limit anything in any way so I fail to see any connection with it and you offering help. We could certainly use an experienced Mac programmer - but the good ones don't start pointless discussions but typically provide patches and we go from there (quite successfully).
Cheers,
Simon
I guess this discussion is going nowhere. Since I am always wrong I give up.
I say having both R.app, R64.app is doubtful (talking about apps from users perspective), you justify that with dumps of some GCC directories from Xcode developer tools (talking about developer tools directories from developer perspective). Can you find me single other app for Mac that comes as two .app packages for each architecture from one install?
I say installing R in 64-bit Linux and launching R GUI app launches 64-bit version (talking how Linux hides its internals), you say I am wrong because you can launch 32-bit app using --arch param. Yes you can, does it prove it is wrong what I said? Typing "R" in OSX command line also launches R in 64-bit, why not 32-bit then?
I say 64-bit code is faster (haven't used "always", but intentionally used word "code" not "program" or "libraries", because these can be badly ported to 64-bit due old compiler or 32-bit only hand optimized assembly code not working for 64-bits, I refer to the machine code that has more wider registers than in 32-bit mode), you say it is not; because it can be slower of faster depending on task, huh :/
You say 64-bit code always use more memory, then you've probably read that: http://software.intel.com/en-us/blogs/2010/07/01/the-reasons-why-64-bit-programs-require-more-stack-memory/
Altogether I am just plain wrong :)
Thank you,
On Wednesday, December 14, 2011 12:42:32 PM UTC-8, Adam Strzelecki wrote:
I guess this discussion is going nowhere. Since I am always wrong I give
up.
I say having both R.app, R64.app is doubtful (talking about apps from
users perspective), you justify that with dumps of some GCC directories
from Xcode developer tools (talking about developer tools directories from
developer perspective). Can you find me single other app for Mac that comes
as two .app packages for each architecture from one install?
I say installing R in 64-bit Linux and launching R GUI app launches 64-bit
version (talking how Linux hides its internals), you say I am wrong because
you can launch 32-bit app using --arch param. Yes you can, does it prove it
is wrong what I said? Typing "R" in OSX command line also launches R in
64-bit, why not 32-bit then?
I say 64-bit code is faster (haven't used "always", but intentionally used
word "code" not "program" or "libraries", because these can be badly ported
to 64-bit due old compiler or 32-bit only hand optimized assembly code not
working for 64-bits, I refer to the machine code that has more wider
registers than in 32-bit mode), you say it is not; because it can be slower
of faster depending on task, huh :/
You say 64-bit code always use more memory, then you've probably read
that:
http://software.intel.com/en-us/blogs/2010/07/01/the-reasons-why-64-bit-programs-require-more-stack-memory/
Altogether I am just plain wrong :)
Thank you,
--
Adam Strzelecki
I think what Simon was trying to point out is that the GUI is split into
R.app and R64.app because otherwise there is no easy way to specify if the
R interpreter embedded in the GUI should start in 32-bit mode or 64-bit
mode similar to the `--arch` switch provided by command line R.
So, the separation is intentional and serves the function of allowing users
who strongly prefer the GUI to choose in which mode they want to run R. As
stated previously in this thread, there are several valid reasons for
running R in 32-bit mode even though the OS is x86_64 native.
-Charlie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20111214/36e8c226/attachment.html>
G'day,
I understand almost none of this (I deal with dogs and cows normally) but historically I had some R packages that would only work 32 bit (RPgSQL was one) , so it was handy to be able to run both 32 and 64 bit instances concurrently. It seems less of an issue now for me (but I'm on a relatively new machine and now use RPostgreSQL!) and most things run in 64 bit.
I guess it's more about R than Mac programming.
I'll shut up and go back to being thankful that (for me as an epidemiologist) it just works.
cheers
Ben
On 15/12/2011, at 10:56 AM, Simon Urbanek wrote:
On Dec 14, 2011, at 3:42 PM, Adam Strzelecki wrote:
I guess this discussion is going nowhere. Since I am always wrong I give up.
I would consider understanding/learning a more productive approach than surrender, but it's up to you ;).
From the beginning you were confusing several entirely unrelated topics (plus unrelated subject), so I'll try for the last time to make it more clear and separate them properly and address them again:
a) 3-way fat R.app (your proposal) vs 2-way fat R.app + 1-way R64.app (CRAN release).
This is entirely a choice of convention and we have decided to separate 32-bit binaries from the 64-bit binary for convenience of the user. As I said there are several reason for this. Whether you like it or not is quite irrelevant - it may not be what Safari does but then Safari is not a stat computing language. Whether you use lipo to add the 64-bit binary to R.app or not is purely cosmetic, it doesn't change the fundamental functionality (except that with R.app+R64.app you can run them in parallel and start the desired architecture more easily - and the user knows what he gets and doesn't need to guess - less relevant for Safari, more so for R).
b) multi-lib approach in R itself: (for packages, modules etc.). This is deliberate and I was illustrating that it is very common even in OS X as you can see in the gcc example. The main reason is portability because this way the build system needs to be implemented only once for all platforms. This is hidden from the user so you should not care. If you do, you may need to read up on it.
c) decision to run 32-bit vs 64-bit binaries. This we leave entirely to the user. You may not be familiar with the details (the link you quoted below is quite irrelevant in the sense that the biggest waste of memory is not on the stack - unless you use deep recursion), but each of the architectures has its weaknesses and advantages. For low-memory machines it is often advisable to use 32-bit binary as the benefits of the 64-bit instruction set are really not that big (and the numerical stability is different, some would say worse). 64-bit binary *always* uses more memory, by definition, that's not even a question. The question is only how much more, and in some practical settings it can be close to 100% more. But, again, we leave this to the user.
d) "How to make Mac 64-bit version feature complete?" Well, the CRAN 64-bit version of R *is* feature complete so the question goes off an invalid assumption. Later you mentioned rgl, but that is not part of R.
I say having both R.app, R64.app is doubtful (talking about apps from users perspective), you justify that with dumps of some GCC directories from Xcode developer tools (talking about developer tools directories from developer perspective). Can you find me single other app for Mac that comes as two .app packages for each architecture from one install?
I say installing R in 64-bit Linux and launching R GUI app
There is no such thing on Linux. R.app exists only on OS X. Windows has Rgui and it has two of them as well (32-bit and 64-bit) and they also come with two separate icons, same as on OS X. Go figure :).
launches 64-bit version (talking how Linux hides its internals), you say I am wrong because you can launch 32-bit app using --arch param. Yes you can, does it prove it is wrong what I said? Typing "R" in OSX command line also launches R in 64-bit, why not 32-bit then?
Because this decision is made at install time depending on your OS X version - it is 32-bit for Leopard, for example. Whether this is the right thing to do is up for discussion (when installed from source you'll get the last installed architecture). R.app doesn't need to make that decision, since it is always 32-bit - easy.
I hope that helps understanding it a bit better. If not, please, consider learning more about R.
Cheers,
Simon
I say 64-bit code is faster (haven't used "always", but intentionally used word "code" not "program" or "libraries", because these can be badly ported to 64-bit due old compiler or 32-bit only hand optimized assembly code not working for 64-bits, I refer to the machine code that has more wider registers than in 32-bit mode), you say it is not; because it can be slower of faster depending on task, huh :/
Yep. If you know a bit more about CPUs, you'll realize that we width of the register is quite irrelevant until you actually use it - which is not very often the case (there are very few actual 64-bit operations). The most time you use it is with pointers where you would have used 32-bit on x86 anyway so you don't gain anything, rather to the contrary. You can gain from having more registers, but only for functions with larger number of arguments (register passing) or functions that are complex enough (fn body). Also larger objects (due to double the size of pointers) mean more very slow operations (out-of-cache memory I/O) and thus slower performance. Most gains you see are in fact due to something different - the fact that x86_64 CPUs can be assumed to have extensions that some old x86 CPUs did not have (various SIMD instructions etc.) which can be used instead of the FPU.
Even if you don't know much about Intel CPU architecture, just run some benchmarks in R - you'll see that it goes either way - you'll have tasks that are slower in 64-bit and you'll have other tasks that are faster in 64-bit (you may be able to dig up examples from this list). It really depends on things like the memory I/O load + data size, types of vectors you operate on, proportion of interpreted code etc. Since R is an interpreter, you'll see bigger negative impact due to increased 64-bit pointer size than you would see in native code (the most vulnerable are generic vectors, lists, environments and character vectors - which are incidentally the most used objects).
Simon,
First of all I really appreciate your responses and of course the work you've done. I may not completely agree with some design decisions, but anyway it is great to have such nice GUI for Mac version of R - IMHO the best GUI of all platforms.
a) 3-way fat R.app (your proposal) vs 2-way fat R.app + 1-way R64.app (CRAN release).
This is entirely a choice of convention and we have decided to separate 32-bit binaries from the 64-bit binary for convenience of the user. As I said there are several reason for this. Whether you like it or not is quite irrelevant - it may not be what Safari does but then Safari is not a stat computing language. Whether you use lipo to add the 64-bit binary to R.app or not is purely cosmetic, it doesn't change the fundamental functionality (except that with R.app+R64.app you can run them in parallel and start the desired architecture more easily - and the user knows what he gets and doesn't need to guess - less relevant for Safari, more so for R).
You're looking on it from developer's perspective. But regular user that want to make few plots via R do not necessarily know what is 64-bit at all. That's why on Mac there's a convention to use FAT binaries and default to 64-bit unless you specify otherwise (like via Finder properties for an app) or there's no capable CPU. All this are for sake of consistency and single experience for all apps.
This is what makes Mac/OSX different from Windows for example - conventions like similar UI for all apps, same keyboard shortcuts for same tasks, same places where every app places its configuration files. So IMHO every developer should ask himself when there comes an idea of making things "better" or "different" that break convention, whether it will really make things better or will lead to confusion.
Just to be clear, I do agree with you in general, that R user should be able to choose between 32-bit and 64-bit version, but this can be done without breaking the OSX conventions, via Finder. Probably I shouldn't be so picky about Mac's R GUI because it is really great, but I just feel it isn't right when some apps behave "different", like placing files in folders I do not expect them to do, etc. etc.
b) multi-lib approach in R itself: (for packages, modules etc.). This is deliberate and I was illustrating that it is very common even in OS X as you can see in the gcc example. The main reason is portability because this way the build system needs to be implemented only once for all platforms. This is hidden from the user so you should not care. If you do, you may need to read up on it.
Sorry but I have to say that. You GCC example is just wrong. GCC is compiler and the folders you mention /usr/lib/gcc/*/4.2.1 are the program DATA not executable code, so these x86_64 folders inside have nothing to do to GCC program executable code itself (i.e. in ARM GCC toolchain these can refer to ARM architecture, while the compiler can run on Intel). GCC executable code as shown via `file /usr/bin/gcc` or `file /usr/lib/libgcc_s.1.dylib` is FAT binary, so Apple does not make any exception here. They do use FAT binaries everywhere.
c) decision to run 32-bit vs 64-bit binaries. This we leave entirely to the user. You may not be familiar with the details (the link you quoted below is quite irrelevant in the sense that the biggest waste of memory is not on the stack - unless you use deep recursion), but each of the architectures has its weaknesses and advantages. For low-memory machines it is often advisable to use 32-bit binary as the benefits of the 64-bit instruction set are really not that big (and the numerical stability is different, some would say worse). 64-bit binary *always* uses more memory, by definition, that's not even a question. The question is only how much more, and in some practical settings it can be close to 100% more. But, again, we leave this to the user.
I never said that 64-bit version use less memory. But saying that 64-bit version can have twice memory usage than 32-bit version is kind of over interpretation. For sure it will (nearly) double its executable size, and also stack memory, but heap (which occupies I believe ~95% memory space in most of uses) won't grow much unless you use memory structures that are made only from pointers. All other types do not change their size in 64-bit, doubles, ints remain same size, at least for AMD64 (not talking here about Sparc or PPC64 :>)
As for memory please also not that 32-bit app 4GB addressing limit is in fact much less, rather close to ~1-1.5GB, because of two factors: VM address space fragmentation, address space range for kernel.
d) "How to make Mac 64-bit version feature complete?" Well, the CRAN 64-bit version of R *is* feature complete so the question goes off an invalid assumption. Later you mentioned rgl, but that is not part of R.
Yes I did false assumption that there's something wrong with 64-bit version so there's separate R64.app. I am sorry about that - renaming the subject of this discussion to something more appropriate.
I say installing R in 64-bit Linux and launching R GUI app
There is no such thing on Linux. R.app exists only on OS X. Windows has Rgui and it has two of them as well (32-bit and 64-bit) and they also come with two separate icons, same as on OS X. Go figure :).
Sorry for the mental shortcut, I know that there's no official R GUI app for Linux, but there's "R --gui=". Still on Linux you won't know any arch distinction exists unless you dig deeper into command line params. On Mac you do, because hey, there are TWO apps installed by the packager. Which once shall I use then? Which one's better? :P
Because this decision is made at install time depending on your OS X version - it is 32-bit for Leopard, for example. Whether this is the right thing to do is up for discussion (when installed from source you'll get the last installed architecture). R.app doesn't need to make that decision, since it is always 32-bit - easy.
No it is not, because the convention is violated. If you run Activity Monitor.app on SL or Lion you will see most of apps are 64-bit, but they don't have any "64" suffix or whatsoever. Such internals are hidden from the Mac user. Apple has decided so, and I do agree with them (even I do not agree with them in many other cases ;)), an average user does not need to know about CPU architectures, so it is OS that should choose for him, leaving of course some way to change that decision.
I hope that helps understanding it a bit better. If not, please, consider learning more about R.
I did one thing wrong, started this discussion with wrong subject bringing a suggestion that R is somehow incomplete. I am very sorry about that, I had no indention to suggest anything like that.
Yes I do learn R, and I find it very mature and advanced software. I do have also MatLab and Mathematica (university licenses) here but I find R superior to them, as it less cluttered, better optimized and what's more important more consistent.
Best regards,
You're looking on it from developer's perspective. But regular user that want to make few plots via R do not necessarily know what is 64-bit at all. That's why on Mac there's a convention to use FAT binaries and default to 64-bit unless you specify otherwise (like via Finder properties for an app) or there's no capable CPU. All this are for sake of consistency and single experience for all apps.
I'd like to chime in. I *am* a regular user. I *do want* to know whether I am using R in 32 or 64-bit mode. I *am* using R, which means that I am *not* Joe random, but a sample of one from a nonrandom selfselected population of people who use a modestly user unfriendly statistical alnguage, either for data analysis or for writing their own code. So, considering R users as the average Mac users is a *wrong* assumption.
Just to be clear, I do agree with you in general, that R user should be able to choose between 32-bit and 64-bit version, but this can be done without breaking the OSX conventions, via Finder.
I damn well do already, clicking on 'R64' rather than 'R', which is the 32-bit R.app. You seem to have a personal beef with the fact that the R GUI does not behave the way *you* think it should. I do have a beef with the quality of the icon of the R.app, so I did make a higher resolution for myself and sent a copy to Simon in case he chooses to use the damn thing. If Simon does not use the hi res icon, I can change the icon of the R.app every time, at infinitum, and live with it.
If you are not happy with the R/R64 situation, write your own GUI and use it. There are other R GUIs for OSX already ("R commander"?, something like that). I'd much rather be able to change the default font permanently in the R.app that get bothered by the 32/64-bit issue.
Just a though.
F
--
Federico C. F. Calboli
Neuroepidemiology and Ageing Research
Imperial College, St. Mary's Campus
Norfolk Place, London W2 1PG
Tel +44 (0)20 75941602 Fax +44 (0)20 75943193
f.calboli [.a.t] imperial.ac.uk
f.calboli [.a.t] gmail.com
If you are not happy with the R/R64 situation, write your own GUI and use it. There are other R GUIs for OSX already ("R commander"?, something like that). I'd much rather be able to change the default font permanently in the R.app that get bothered by the 32/64-bit issue.
The reason I started this thread was not to express my unhappiness, but simply asking why there are two R apps installed by the package. So that was just a curiosity. Since I do use "rgl" a lot, which falls back to X11 on 64-bit rather than native UI, I figured out (wrong) that this may be the reason. I work with pretty heavy data, have 12GB RAM, so going 64-bit was natural choice for me, but going trough X11 for OpenGL is not what I'd expect. That's why I am going to make Cocoa wrapper for rgl that should handle that.
There is nowhere any complaint stashed in what I wrote about how R works or how it looks like (the icon is IMHO nice), I was just asking for the REASON and PURPOSE having two apps. It is just Simon's arguments do not convince me, despite my regard for him. You can switch between arch via Finder, you get a report about current arch during R startup. But you guys take what I wrote (which is just an opinion) as accusations. As I said the only mistake was the original subject that was suggesting some "incompleteness" based on false assumption related to "rgl", and I am sorry about that.
Just like that.
Cheers,
On Dec 15, 2011, at 10:52 AM, Adam Strzelecki wrote:
If you are not happy with the R/R64 situation, write your own GUI and use it. There are other R GUIs for OSX already ("R commander"?, something like that). I'd much rather be able to change the default font permanently in the R.app that get bothered by the 32/64-bit issue.
The reason I started this thread was not to express my unhappiness, but simply asking why there are two R apps installed by the package. So that was just a curiosity. Since I do use "rgl" a lot, which falls back to X11 on 64-bit rather than native UI, I figured out (wrong) that this may be the reason. I work with pretty heavy data, have 12GB RAM, so going 64-bit was natural choice for me, but going trough X11 for OpenGL is not what I'd expect. That's why I am going to make Cocoa wrapper for rgl that should handle that.
There is nowhere any complaint stashed in what I wrote about how R works or how it looks like (the icon is IMHO nice), I was just asking for the REASON and PURPOSE having two apps. It is just Simon's arguments do not convince me,
You asked about reasons, you got them. They are not intended to convince you. I even told you how you can make yourself happy - but note that it is about *you*, not anyone else. You have the right to disagree, but you don't have the right to impose your view on everyone else (unfortunately a more common trait in recent times) - which is what the exchange looked more like. (More off-list.)
Cheers,
Simon
despite my regard for him. You can switch between arch via Finder, you get a report about current arch during R startup. But you guys take what I wrote (which is just an opinion) as accusations. As I said the only mistake was the original subject that was suggesting some "incompleteness" based on false assumption related to "rgl", and I am sorry about that.
Just like that.
Cheers,
--
Adam Strzelecki
On Dec 14, 2011, at 11:38 AM, Timothy Bates wrote:
On 14 Dec 2011, at 4:24 PM, Adam Strzelecki wrote:
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture. So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
I?ll second that: It makes AppleScripting easier too: Calling ?R.app? always works, instead of every user having to customise scripts with their app version R64.app vs R.app
That is fair enough. Do other users have an opinion either way?
I work around this by deleting one binary and renaming R64 to R.app
You can simply run
lipo -create /Applications/R.app/Contents/MacOS/R \
/Applications/R64.app/Contents/MacOS/R \
-o /Applications/R.app/Contents/MacOS/R
That will create the 3-way universal R.app as discussed.
Cheers,
Simon
On Dec 14, 2011, at 11:38 AM, Timothy Bates wrote:
On 14 Dec 2011, at 4:24 PM, Adam Strzelecki wrote:
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture. So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
I?ll second that: It makes AppleScripting easier too: Calling ?R.app? always works, instead of every user having to customise scripts with their app version R64.app vs R.app
That is fair enough. Do other users have an opinion either way?
(Using Mac OS X 10.6.8)
Most of the time I use R 32-bit.
Currently my machine has 4Gb memory so 64-bit is not needed.
I have found that quite often 32-bit R runs faster than the 64-bit version.
In Terminal on the commandline we have R32, R64 and R (which is 64-bit).
So to have R.app be 32-bit looks a bit odd.
I don't like the Finder way to make applications start in 32-bit or 64-bit mode (Get Info and check a box).
Also for a user with no permissions to fiddle in /Applications this would make life difficult.
My suggestion would be the following:
R32.app ==> runs 32-bit R
R64.app ==> runs 64-bit R
R.app could be the main application responding to applescript "R.app", defaulting to 64-bit.
And R32 and R64 somehow start R in 32-bit and 64-bit mode respectively.
Or R32 and R64 are separate applications both responding to "R.app" (this precludes both running simultaneously?).
And R.app an alias to one of the above.
Hopefully this makes sense.
Berend
I work around this by deleting one binary and renaming R64 to R.app
You can simply run
lipo -create /Applications/R.app/Contents/MacOS/R \
/Applications/R64.app/Contents/MacOS/R \
-o /Applications/R.app/Contents/MacOS/R
That will create the 3-way universal R.app as discussed.
Cheers,
Simon
On Dec 16, 2011, at 11:20 AM, Simon Urbanek wrote:
On Dec 14, 2011, at 11:38 AM, Timothy Bates wrote:
On 14 Dec 2011, at 4:24 PM, Adam Strzelecki wrote:
In addition, there is no benefit in creating universal binaries,
since they are very Darwin-specific and bring no benefit in this
context.
There's a huge benefit of doing that. I do develop Mac apps, and
FAT binaries and libs makes the other apps referring to them to
refer to single file path regardless of architecture. So this is
benefit for developers. Another benefit is for Mac users, that
they run single app regardless if they run on PPC, i386 only Intel
(first Intel Macs ?!) or latest 64-bit Macs.
I?ll second that: It makes AppleScripting easier too: Calling
?R.app? always works, instead of every user having to customise
scripts with their app version R64.app vs R.app
That is fair enough. Do other users have an opinion either way?
I've never had any confusion at that level of using R.app or R64.app.
I was really grateful that R64 was there when I needed it. There was
some confusion about package availability for the 64-bit version a
couple of years ago, but that seems to have faded away.
I work around this by deleting one binary and renaming R64 to R.app
You can simply run
lipo -create /Applications/R.app/Contents/MacOS/R \
/Applications/R64.app/Contents/MacOS/R \
-o /Applications/R.app/Contents/MacOS/R
That will create the 3-way universal R.app as discussed.
I don't read system commands very well. On my laptop with a hardware
restriction to 4GB will that load R64.app if it's not needed?
On Dec 16, 2011, at 12:50 PM, David Winsemius wrote:
On Dec 16, 2011, at 11:20 AM, Simon Urbanek wrote:
On Dec 14, 2011, at 11:38 AM, Timothy Bates wrote:
On 14 Dec 2011, at 4:24 PM, Adam Strzelecki wrote:
In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture. So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
I?ll second that: It makes AppleScripting easier too: Calling ?R.app? always works, instead of every user having to customise scripts with their app version R64.app vs R.app
That is fair enough. Do other users have an opinion either way?
I've never had any confusion at that level of using R.app or R64.app. I was really grateful that R64 was there when I needed it. There was some confusion about package availability for the 64-bit version a couple of years ago, but that seems to have faded away.
I work around this by deleting one binary and renaming R64 to R.app
You can simply run
lipo -create /Applications/R.app/Contents/MacOS/R \
/Applications/R64.app/Contents/MacOS/R \
-o /Applications/R.app/Contents/MacOS/R
That will create the 3-way universal R.app as discussed.
I don't read system commands very well. On my laptop with a hardware restriction to 4GB will that load R64.app if it's not needed?
Yes (practically). The above "lipo" command merges the 64-bit binary from R64.app into R.app (and thus R64.app becomes superfluous). By default this would make OS X 10.6+ pick 64-bit on machines that support it, even if they don't need it. You can override that behavior in the Finder, though. I'm not suggesting that it's a good thing, it's for those that would prefer just a single R.app for everything - which is what sparked this discussion.
Cheers,
Simon
Just to give you some numbers, on my iMac Lion 10.7.2, i5 Sandy Bridge.
? When allocating matrix 10000x10000 R64 takes 798MB vs R32 taking 785MB, so 14MB is wasted for 64-bit handling
? Running R-benchmark-25.R from http://r.research.att.com/benchmarks/ in R64 renders about ~5% performance gain
I am aware these number can really differ on older OSX, notably on Leopard, also 64-bit performance can be very bad for 64-bit using vecLib on older OSX, vecLib was somehow borked by Apple.
On other side Apple depreciates slowly 32-bit in newer systems, in Lion there's no 32-bit kernel anymore, most of built-in apps are 64-bit only. Still is can run 32-bit apps, but it cannot run PPC anymore (no Rosetta).
So IMHO the picking 64-bit on 10.6+ (or 10.7+) is good option for the future, as the scientific datasets grow every year and RAM installed by default by Apple will be soon min 8GB.
(Some tech blather: 64-bit program can allocate in fact more than 4GB when having only 4GB RAM - implies paging and swapping, unless allocated pages remain non-written. In theory 64-bit program can allocate few 1TB on 4GB RAM system, with no physical memory usage bump. Memory will be used only when user will 1st access the allocated pages.)
Regards,
That is fair enough. Do other users have an opinion either way?
Yes, indeed: I find it very convenient to be able to choose whether I run 32-bit or 64-bit R.app (for the same reasons that have already been brought forward by others) and to run both flavours in parallel.
Although I've been using Mac OS X exclusively for more than 6 years, I wasn't aware of the fact that I can start a fat app in 32-bit or 64-bit mode through Finder. I doubt that inexperienced users -- those most likely to have a less powerful machine that works better with 32-bit R -- will know about this possibility, or ever find out. Also, Adam, can you please show us how to select 32-bit vs. 64-bit while launching R.app from the dock or through Quicksilver etc. (which is what I always do)?
Cheers,
Stefan
Some tech blather: 64-bit program can allocate in fact more than 4GB when having only 4GB RAM - implies paging and swapping, unless allocated pages remain non-written.
And that's a very good reason why I prefer to have an easy way to launch 32-bit R.app: it keeps my MacBook Pro (which has only 6GB RAM) from locking up due to R using too much memory (of course, R will crash or abort the script, but I prefer this very much to not being able to work with the computer at all ...).
Cheers,
Stefan
And that's a very good reason why I prefer to have an easy way to launch 32-bit R.app: it keeps my MacBook Pro (which has only 6GB RAM) from locking up due to R using too much memory (of course, R will crash or abort the script, but I prefer this very much to not being able to work with the computer at all ...).
(?)
Yes, indeed: I find it very convenient to be able to choose whether I run 32-bit or 64-bit R.app (for the same reasons that have already been brought forward by others) and to run both flavours in parallel.
(?)
Also, Adam, can you please show us how to select 32-bit vs. 64-bit while launching R.app from the dock or through Quicksilver etc. (which is what I always do)?
I cannot, because there's no way I know. Your points are absolutely right, and since you find switching between R.app & R64.app convenient, it makes no sense to place any techie arguments against other users' habits.
Altogether as long as "'memory.limit()' is Windows-specific" it indeed makes sense to keep users pinned by default to 32-bit, because a slip of the finger on extra 0 during matrix allocation probably can blow your Mac's memory :)
Regards,