Skip to content

Packages and Libraries (was: Re: lme4 "package" etc ..)

20 messages · Martin Maechler, (Ted Harding), Achim Zeileis +6 more

#
tony> For OBVIOUS reasons, is there any chance that we could introduce
    tony> "package()" and deprecate "library()"?

This idea is not new {as you must surely have guessed}. In fact,
there's a much longer standing proposition of  "usePackage()"
(IIRC, or "use.package()" ?).  However, we (R-core) always had
wanted to also provide a ``proper'' class named "package" 
along with this, but for several reasons didn't get around to it.. yet.

-- I've diverted to R-devel now that we are really talking about
   desired future behavior of R

    tony> (well, I'll also ask if we could deprecate "=" for assignment, but
    tony> that's hopeless).
:-)


    tony> On Tue, 8 Feb 2005 11:49:39 +0100, Martin Maechler
tony> <maechler@stat.math.ethz.ch> wrote:
>> >>>>> "Pavel" == Pavel Khomski <pkhomski@wiwi.uni-bielefeld.de>
    >> >>>>>     on Tue, 08 Feb 2005 10:20:03 +0100 writes:
    >> 
    Pavel> this is a question, how can i specify the random part
    Pavel> in the GLMM-call (of the lme4 library) for compound
    Pavel> matrices just in the the same way as they defined in
    Pavel> the lme-Call (of the nlme library).
    >> 
    >> ``twice in such a short paragraph -- yikes !!'' ... I'm getting
    >> convulsive...
    >> 
    >> There is NO lme4 library nor an nlme one !
    >> There's the lme4 *PACKAGE* and the nlme *PACKAGE* -- please --
    >> 
    >> ....................
#
But I don't see a problem with "package("package")", though I'm sure
I'm missing something.

It really would end this constant confusion and save various folks
approx 15 minutes/week in knee-jerk responses, eh?

best,
-tony


On Tue, 8 Feb 2005 15:09:42 +0100, Martin Maechler
<maechler@stat.math.ethz.ch> wrote:

  
    
#
package() [sic] might be the creator for package objects, provided we
can decide on what they are (and what kind of packages [source,
installed, ...] they are used for).

usePackage() or use_package() otoh would indicate to "use" a package
(i.e., load and attach it).  The tricky part is deciding about the
interface (e.g., finally disallowing non-standard evaluation as it is a
programmer's nightmare) and what it should return.  And that is work in
progress ...

Even if we don't like the current semantics, the *name* of library() in
itself should not be a problem.  After all, calling summary() does not
imply that your primary argument is a summary, so why should calling
library() imply that its primary argument is a "library"?
Afaic, one of the issues is that it seems common practice to refer to
collections of code as "libraries" or "packages", and we're trying to
use these rather general-purpose terms in a very precise meaning, and
obviously not very successful, in particular because the use of
"library" is highly non-standard.  One idea might be to replace the
"library" by something else like ... "a place where R knows where to
find packages" ... hmm, now that's too long, so ...

-k
#
On Tue, 8 Feb 2005, A.J. Rossini wrote:

            
Grammar.  It's like `R CMD build' which some people think installs: here 
you don't want `to package' you want `to use', and these are read as 
imperatives.

I think we are all agreed that some variant on usePackage() would be good, 
but here the perfect is the enemy of the good and the good the enemy of 
the adequate.

Let's see if someone cares enough to push a design through for 2.1.0.
#
On 08-Feb-05 Kurt Hornik wrote:
The only issue I have with "library" vs "package" (or vice versa)
is the following.

I *know* that a collection of functions etc. (such as nlme) is
properly called a "package". I could (and do) get ticked off if
I refer to it as a "library".

However, if I want to use it then I enter

  library(nlme)

As a result, the little daemons who potter around re-wiring
my mental circuits sometimes cross-connect these two. As a
result, I can inadvertently refer to "the nlme library".

An example is a recent mailing of mine to the list where
I twice referred to a certain "library" (and was amiably
rebuked by Martic Maechler) -- but then I saw that I had
also twice referred to the same "package" in the same mail!

Personally I don't mind whether it is called "package" or
"library", though using the command library() to load a
package does tend, as described, to get me treading on my
own shoelaces.

But, a propos, is there an R entity called a "library"
(other than the command) as distinct from a "package"?

If so, then I can accept the necessity for disciplined
distinction between them, since calling one sort of thing
by the name of another sort of thing necessarily creates
confusion. But if not, then I'm inclined to feel that
a bit of loose talk is acceptable (since then a reference
to a "library" would be understood as a reference to a
"package", since what else would it be?)

Just my thoughts ...

Ted.


--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding@nessie.mcc.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 08-Feb-05                                       Time: 22:19:46
------------------------------ XFMail ------------------------------
#
On Tue, 08 Feb 2005 22:19:46 -0000 (GMT) (Ted Harding) wrote:

            
A *library* is a directory in which you can find R *packages* (just as
in real life you can find books in a library) and with 
  library("foo", lib.loc = "/path/to/bar")
you want to get the package (book) "foo" from the library "bar" located
at "/path/to/bar".

So the two are really distinct...in real life, you also wouldn't say
that you have been in the book where they had a lot of libraries on the
shelves, would you? ;-)

But as Kurt explained: this distinction between "library" and "package"
is specific to R and does not correspond to common practice for other
software systems.
Z
#
On Tue, 8 Feb 2005 Ted.Harding@nessie.mcc.ac.uk wrote:
[...]
Yes.  That is what the argument 'lib.loc' to library() and other functions 
refers to.

A 'library' is a collection of packages stored in one directory.

library("pkg") means `go to one or more libraries, find package pkg and 
load it up'

In S parlance what in R is a `package' is a `library section' or 
(latterly) also `chapter'.

In English usage a library is a collection of volumes, not of recipes or 
stories or articles.
#
On Wed, 9 Feb 2005, Achim Zeileis wrote:
Out of pure curiosity, could anyone tell me the historical reason that
library() is used here?  Does it tie in to the S ancestry of R?
#
On Wed, 9 Feb 2005 00:02:53 +0100, Achim Zeileis
<Achim.Zeileis@wu-wien.ac.at> wrote:
Sure, but I'm not specifying which library I'd like to get the
packages from, thus it might be make perfect sense to say:

loadPackage("foo",library="bar")

But when I'm typing library, I'm loading a package, NOT specifying a
library to use.

Package is also confusing -- I keep thinking of it as a noun, as a
package specifier, while others like it as a verb; this was a useful
point for me that one person made.

But perhaps then adding the verb (well, I can see the point against,
but hear it out) might be useful.  Thus,

    package(action,location,name)

might be a useful function signature, though possibly not in that order.
#
On Tue, 8 Feb 2005 23:05:20 +0000 (GMT), Prof Brian Ripley
<ripley@stats.ox.ac.uk> wrote:
And this is the problem, that the description doesn't clearly match
the specification.

One might easily expect that library("pkg") implies use library "pkg"
for further package loading.

It's almost too bad that libraries weren't books, with packages being chapters, 
or libraries being postOffices, with packages being packages,
or packages being libraries, with libraries being cities
or libraries being libraries, with packages being books.

But the current mess wastes a good bit of time aggravating people who
want things just so, responding to people who are just careless.
#
As I wrote earlier, we need to have alternatives to change this.

The R system is highly extensible through standardized add-ons called
*packages*.  That is one of its key strengths, and I don't think we
should stop referring to packages as packages.

Packages are made available by putting them into *libraries*, defined as
"places where R knows to find packages".  If we do not like this term,
we need a better one.

Packages are loaded and attached using library(), which in principle is
something for which a replacement is desired anyways.  But as Brian and
I [at least] said, we need not only a new name, but also a careful
redesign, and someone taking charge.

-k
#
On Tue, 8 Feb 2005, Jeff Gentry wrote:

            
It's been the way S does it since ca 1987 (when the Blue Book version of S 
first made S extensible via functions as today).  See the 1988 Blue Book 
p.58.  The only difference (as I have already noted) is that the S library 
has `sections' not `packages'.
#
In my recent experience, a post office is somewhere that packages are 
*lost* rather than found :)

Baz

[I had a card from the PO saying they'd tried to deliver a package but I 
was out, I went to the depot and they couldn't find it...]
#
Kurt Hornik <Kurt.Hornik@wu-wien.ac.at> writes:
Any information on the rate...? (I still vote for usepackage() btw.)

It would be good if we could at least have an outline of the intended
functionality and see if we could forge ahead and get a preliminary
version done in time for 2.1.x
More likely it would imply that the *result* is a library...

Anyway, it was introduced at a time where we considered it important
to be "prototype compatible" as long as there was no good reason not
to. With 20-20 hindsight, we could probably have afforded to think up
a better name.
"store", or "depot" springs to mind. The latter might cause baz to go
postal, though...
#
Why not use(), as the GCD?
Help us out.

	use <- function(package, pos = 2, lib.loc, ...)

where 'package' is either a character string or some sort of package
object/reference, to be specified later.  And 'lib.loc' needs to have a
different name if we rename libraries into stores or whatever ...

What should this return?  Currently, 'library' returns the list of
loaded (or available) packages by default, as a list of names, which is
not good enough.  So we need something like the DLLInfoList returned by
getLoadedDLLs() (and the docs should actually mention that class), or
something usable by the package management tools ... and this is under
redesign as well.

But why should this really return info on all loaded/attached packages?
An alternative might be just returning the package meta-data in some
form.  Or nothing, which would fit into the idea that it really does
nothing apart from loading and attaching a package.

(And maybe a condition object inheriting from packageLoadAndAttachError
in case of failure? :-))

-k
#
On Wed, 9 Feb 2005 17:14:08 +0100, Kurt Hornik
<Kurt.Hornik@wu-wien.ac.at> wrote:
Excellent suggestion, Kurt.
use <- function(packageName,pos=2,library, ...)

I could argue that "library" and "lib.loc" try to describe the same
thing (a name and its pointer).
I think package ought to be a character string.   Unless you want to
combine the packageName and libraryLocation into some form of data
object, or packageName, libraryLocation, and an environment containing
the erstwhile contents?
Perhaps "use" should incorporate "require" functionality, i.e. TRUE or
FALSE depending on whether you can use it after the "use" function
call.
I like "libraryContents()" or similar to figure out loaded and
potentially loadable packages.
Yes.  whatever.
#
And so obvious :-)
We need to decide whether to stay with the "library" concept, or maybe
go for another one alongside.
For the time being, the package would definitely be specified as a
character string with its name.  But [RG will like this] I find it
rather clumsy to have tons of equal-level arguments which in fact could
use some hierarchical structure.  What we really want is some
SPECIFICATION of the package to be used, definitely including its name,
but maybe also where it should come from (library location) or which
version it should have at least or at most, etc.  So something like

   use(package_spec("foo", version = ~ > "1.5.8"), pos = 12, ...)
As Brian said, we would most likely get rid of library() alongside.

I would condiser the TRUE/FALSE paradigm outdated.  The FALSE case
really consists of a variety or error conditions, and we should return
these as such (rather than indicating "oh there was some problem but
were not telling you which one, unless you are a human reader and look
at the condition message ...").
installed.packages() for the time being.
-k
#
Full agreement.  Thanks, Kurt.


On Thu, 10 Feb 2005 08:53:03 +0100, Kurt Hornik
<Kurt.Hornik@wu-wien.ac.at> wrote:

  
    
#
A.J. Rossini wrote:
use(package_spec("foo", version = ~ > "1.5.8"), pos = 12, ...)

  What else might you 'use'? .RData files perhaps? Dynamic links to 
databases? Is this going to turn into a monster that consumes library() 
and attach()? Not that that might be a bad thing, there's a very fine 
line between them at times...

Baz
#
Conceptually, things than can give RObjectTables as in DTLs package.

-k