On 30/04/2022 2:58 p.m., Patrick Schratz wrote:
They don't go there "silently" as in unnoticed - they go there if
you instruct R to do so. That's why there is an explicit choice in
the Installer. Otherwise regular R rules apply.
Where is this choice in the installer? I don?t see a menu/setting which
users could change to install packages into a user lib instead of the
system lib (if they are part of the |admin| group).
To me, they go there if the lib is writable - and ?the common? R user
does not know that the system lib is writeable by default.
I think Simon is talking about the package installer in R.app and you are talking about the installer for R itself. The package installer dialog in R.app has a pretty clear section called Install Location. Duncan Murdoch
It only does so for admin users. Unlike "managed" unix systems, on
macOS you have essentially two situations:
On a "personal" machine (like laptop) the user is the main user and
admin. Therefore it makes a lot more sense for the user to use a
single location for managing packages which is at the system level.
This also allows easy R upgrades. In addition, locations in user
home raise a lot of issues (see the various discussion where this
bites people on Windows) due to interactions with software that does
mirroring, backups etc.. Hence this approach "just works" as one
would expect on a Mac.
To be clear: I don?t question the system-wide installation and I think
this is good as is (this also happens on Linux).
I am questioning the group write permissions for the system lib.
If the user wishes to use his/her private library, it is trivial -
just click on "At User Level" and from then on all packages are
managed user's local library just like on any other platform.
I might be missing something obvious but where is this option ?at the
user level?? I assumed you?re talking about the official |.dmg|
installer - which does not have such an option AFAICS?
On a "managed" Mac the user is not an admin and therefore the
behavior makes no difference. The status quo just makes it easier
for admins to manage the shared library, but in reality this doesn't
matter as one would assume the admins know what they are doing.
I disagree on this, especially the point ?makes it easier for admins to
manage the shared library?.
Admins should (and will) always be able to manage the system lib /after/
authenticating (as on Linux). The authentication step does not really
make a difference in practice for admins and is required in almost all
places where system-wide changes are desired.
This is also the /core/ of the whole discussion: 775 vs 755 (WRT to
directory permissions).
I don?t see any (strong) reason that would result in 775 > 755.
(If so, then the default should probably also be changed on Linux.)
In a previous message, Kevin Ushey also agreed on the point that
explicit authorization should be required to write into the system lib.
I assume there might be many more people who would actually agree on
this 755 being preferable in this situation.
How can a proposal be phrased to reconsider this setting that is
evaluated by a representative group of people?
I am not claiming to be right but I?d be interested in a multi-person
evaluation of this setting rather than keeping this at a
person-to-person discussion level.
Well, having administered company-wide R installations in large
companies for almost two decades I'd strongly disagree. As an admin
you want as few user-installed packages as possible, because they
are guaranteed to cause problems. You want to limit this for things
like development of packages where you want the stable version
globally and development version locally (and this is not just me -
have a look how the top tech companies manage their software). You
have a reliable, stable central location - if you don't do that then
you'll have n libraries to manage for n users which is absolutely
not sustainable as users will break their libraries and you cannot
even upgrade R. Also having a central, shared library is crucial for
collaboration. Unlike in Python in R it actually works since R and
CRAN doesn't allow randomly breaking reverse-dependencies.
As a system engineer and admin myself (for several ?large?
companies/institutions), I kindly disagree on your view here.
User packages are not a problem but /a feature/, everyone can install
the versions they need for their project.
They don?t interfere with packages from other users and are not forced
go with the update interval of an admin.
With the additional use of renv (thanks Kevin!), redundancy is highly
reduced as a shared cache can be used from with users can simply use
symlinks rather than installing the x-th copy of the same R package
version. But this is partly off-topic WRT to the actual discussion.
Overall this sub-discussion part might come down to the philosophy of
having a ?centrally managed, unflexible admin installation? or a
?centrally managed, partly-flexible admin installation? where only the R
versions and system libs are managed but users have the freedom to
install any R packages they want.
Also in ?my? philosophy, it?s not about ?upgrading? R and removing the
previous version but adding new versions as they come in and keeping
previous ones - for the purpose of reproducibility.
I usually keep the latest patch version of a minor version and aim to
provide a consistent R environment for various minor versions where
users are guaranteed to be able to work with that minor version in a
flexible way (i.e. by installing user packages as they want) for many
years ahead.
As mentioned before and above I disagree. The proposal doesn't
matter for managed Macs but would negatively affect users that are
single-user admins and since that is typically the case for the
majority of Mac R users (as they typically are on their personal
machines) I don't see any upshot. All it would do is to prevent
typical R users to install packages directly.
How would it affect single-user admins in a negative way?
They can
* still install packages per R minor version into a dedicated user library
* install multiple R minor versions side by side
* actually enjoy the same behavior as on Linux
All it would do is to prevent typical R users to install packages
directly.
I don?t understand this point. It would behave similar as on Linux,
where users are prompted to create a user library (on first use and if
non exists yet).
As you can see, the overall discussion topic is quite important to me
and I am still convinced that the current state on macOS is suboptimal.
Thanks for your time and sharing your thoughts.
Cheers
Patrick
On 25 Apr 2022, at 1:46, Simon Urbanek wrote:
Patrick,
sorry fo the delayed reply - this was not a quick e-mail so I had to
find time after the release :)
On Apr 3, 2022, at 8:26 PM, Patrick Schratz
<patrick.schratz at gmail.com> wrote:
Hi Simon,
thanks for your extensive reply.
The choice is deliberate: the admin group on macOS corresponds
to users that are allowed to install system-wide software so it
allows all admins on the machine to install packages which is
the expected way on macOS.
I think this choice is unfortunate as it contrasts with existing
behavior on other platforms where one needs to explicitly
request admin privileges by either using sudo or starting R as
an admin.
On macOS, packages just go into the system lib ?silently?
because of the write permissions granted via the admin group.
See also my comments further down for more details on this.
They don't go there "silently" as in unnoticed - they go there if
you instruct R to do so. That's why there is an explicit choice in
the Installer. Otherwise regular R rules apply.
Also the versioning of the R framework as x.y is also deliberate
- upgrading R to a new patch version does *not* require
re-installation of packages, they work by design so in fact the
system location is the safest way to do that. Also note that
packages are never removed by the installer.
Thanks, I am aware that a patch update does not require a
reinstallation as the packages are functional across minor versions.
I checked again and was indeed wrong, patch updates from the
CRAN installer do not remove additional custom packages in the
system lib.
I was confused by some custom approaches of mine which cause
this behavior - sorry for this!
So out of the items listed in "The problem" they are all not
true with the exception of the comparison with the other
platforms, but even that difference is very subtle as it only
affects the default on the first installation and not regular
use (and I'm, not even sure it that is true since admin users
can still install in the system location on other platforms).
On Linux you would need to do explicitly invoke sudo R to allow
writing into the system lib.
The issue on macOS is that a regular start of R allows system
lib write permissions.
In my current view I think a similar behavior as on Linux would
be great, i.e. to explicitly having to request admin privileges
for this task.
It only does so for admin users. Unlike "managed" unix systems, on
macOS you have essentially two situations:
On a "personal" machine (like laptop) the user is the main user and
admin. Therefore it makes a lot more sense for the user to use a
single location for managing packages which is at the system level.
This also allows easy R upgrades. In addition, locations in user
home raise a lot of issues (see the various discussion where this
bites people on Windows) due to interactions with software that does
mirroring, backups etc.. Hence this approach "just works" as one
would expect on a Mac. If the user wishes to use his/her private
library, it is trivial - just click on "At User Level" and from then
on all packages are managed user's local library just like on any
other platform.
On a "managed" Mac the user is not an admin and therefore the
behavior makes no difference. The status quo just makes it easier
for admins to manage the shared library, but in reality this doesn't
matter as one would assume the admins know what they are doing.
I don?t understand the part ?as it only affects the default on
the first installation and not regular use? of your reply -
could you clarify this?
Unless a user creates a user lib manually, packages will always
go into the system lib - not only on first use - but I might be
misunderstanding your comment here.
I would argue that the current setup tends to be a lot safer
than the alternatives, because it allows commonly used packages
to be installed at the system level and private packages to be
installed at user level. This is also the design typically used
on shared machines, where you separate local packages from user
packages where local ones are installed by administrators - so
exactly the same setup. Moreover R upgrades are a lot cleaner,
since you can easily upgrade all system packages at once so you
don't have to worry about individual users having stale packages
- the biggest problem for admins.
Yes and no.
Sometimes system admins want to install certain packages
globally - however, I never do so for the following reason:
Often this will lead to multiple package installations, i.e. one
in the syslib and one in the user lib (if the user installs the
package again for some reason which quite often happens).
Often these duplicated packages will have different versions and
users are confused which one is actually loaded (the user lib
one is as it is first in the path).
Aside from this practical point, Macs are rarely used in a
shared way.
And even if, I?d highly favor having to request write
permissions into the syslib rather it happening by default.
Imagine a scenario where the admin of a shared Mac constantly
writes into the syslib (because this is the default). This
syslib is then also used by other non-admin users on the system.
I don?t think this is a desired scenario and might cause lot?s
of confusion (not even mentioning the fact if all people in this
scenario are aware what?s going on given that this is a niche
topic).
Here I think a one-time central installation of R and then only
working with user libs (as on Linux) would be preferable.
Well, having administered company-wide R installations in large
companies for almost two decades I'd strongly disagree. As an admin
you want as few user-installed packages as possible, because they
are guaranteed to cause problems. You want to limit this for things
like development of packages where you want the stable version
globally and development version locally (and this is not just me -
have a look how the top tech companies manage their software). You
have a reliable, stable central location - if you don't do that then
you'll have n libraries to manage for n users which is absolutely
not sustainable as users will break their libraries and you cannot
even upgrade R. Also having a central, shared library is crucial for
collaboration. Unlike in Python in R it actually works since R and
CRAN doesn't allow randomly breaking reverse-dependencies.
From a technical perspective, I know that setting root:root on
macOS is not possible. My proposed change to 755 (and leaving
root:admin) would however exactly mimic this (and the one of
Linux installs) behavior:
? admins would need to do sudo R to install into the system library
? otherwise they are prompted to create a user library
Which downsides would this approach have? Currently I don?t see
any. It would even harmonize CRAN installer behavior across
platforms.
I'd be happy to hear from more Mac user if there are reasons to
change the default, but as I outlined the choices were
deliberate after weighting the pros and cons. In my view the
major issue with the proposal it that is would prevent sharing
of packages, make R upgrades a lot harder and prevent admin
users from using the current tools for package management - and
that includes the ability to separate system and user packages
on single-user machines.
I?ll try to vision the practical changes of this:
? Patch update experience would not change as custom packages
will be in the user lib for the respective minor version (by
default)
? Admins are still able to install into the system lib when
using sudo R
? AFAICS admins will still be able to separate system and user
packages as they can use sudo R for syslib installs. To me, the
proposed change would even make the behaviour more clear than
before (which requires to create a hidden folder (user lib) in
the right place to actually use a user lib).
Let me know if I overlook something - but currently I don?t see
any downside but various positive impacts.
As mentioned before and above I disagree. The proposal doesn't
matter for managed Macs but would negatively affect users that are
single-user admins and since that is typically the case for the
majority of Mac R users (as they typically are on their personal
machines) I don't see any upshot. All it would do is to prevent
typical R users to install packages directly.
Last, I wanted to ask if the source code for the CRAN installer
is publicly available? I could not find it and would be
interested to take a look into it. If this is not possible for
some reason, I would also be interested in getting to know the
reason for this decision.
Everything is in the R SVN, the R build and release system is in
https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4
<https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4>
and Apple Installer packaging is in
https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging
<https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging>
and the relevant postflight script is in
https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging/scripts-R-fw/postflight
<https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging/scripts-R-fw/postflight>
On Apr 13, 2022, at 8:43 PM, Patrick Schratz
<patrick.schratz at gmail.com> wrote:
Related to this Q: Are the macOS CRAN policies actively
discussed by a team of people (who might eventually also be
willing to share their thoughts or could be addressed with such
questions) or are you solely responsible for it?
CRAN is an entire team, so yes, but as for anything Mac-related it
includes R-core and other stake holders that have expressed interest
before (e.g. Bioconductor). Obviously, this (R-SIG-Mac) is also a
good place as that includes anyone who cares about R on macOS.
Cheers,
Simon
_______________________________________________ R-SIG-Mac mailing list R-SIG-Mac at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac