Wish list

Let me add to the wish list the creation of some mechanism to better track
improvements and bug fixes in packages, such as a change log link by each
package's area in CRAN, or easy access to CVS information from there. 
When I report bugs (e.g., in read.xport in foreign [due somewhat to
problems inherent with SAS's format] or ace or avas in acepack) it would
be nice to see some announcement when the bugs are resolved, or to easily
track this.  Even a checkbox that the package maintainer has seen the bug
report even if she/he currently does not have time to work on it would be
very helpful, as would a notation that the bug report was found to be
"buggy".

Frank E Harrell Jr writes:

Let me add to the wish list the creation of some mechanism to better track
improvements and bug fixes in packages, such as a change log link by each
package's area in CRAN, or easy access to CVS information from there. 
When I report bugs (e.g., in read.xport in foreign [due somewhat to
problems inherent with SAS's format] or ace or avas in acepack) it would
be nice to see some announcement when the bugs are resolved, or to easily
track this.  Even a checkbox that the package maintainer has seen the bug
report even if she/he currently does not have time to work on it would be
very helpful, as would a notation that the bug report was found to be
"buggy".

	I highly agree with you on this. It would be very nice having a fully
featured bug reporting system, where you could upload patches, discuss
improvements on existing packages or on the R-core itself, request for
features and so on. I think that Bugzilla (www.bugzilla.org) would suit
these expectations very well. It is the bug tracking system used by huge
projects like Mozilla (bugzilla.mozilla.org), Gentoo (bugs.gentoo.org),
and  Redhat (bugzilla.redhat.com), and based on my own experience I'd say
it addresses most of the things you pointed out.

	It works (at least in Gentoo, which is the one I'm more used to
working with) like this: someone files in a bug report. In the bug report
itself one informs the type of the bug report (a bug, a feature improvement, a
request to the developer), the severity, and any other relevant information. It
is also possible to upload attachments (like proposed patches) or additional
information on the report.

	The bug report then is assigned to a given group or, in the case of
packages to the person who is in charge for mantaining it. Anyone then can read
the bug report and make suggestions or propose fixes (see for example:
http://bugs.gentoo.org/show_bug.cgi?id=30784 ). [As opposed to the current
system, where the bug report can't even be linked to a website, and all the
discussion should be done via the mailing lists]. The maintainer or any
other other authorized developer can then accept or reject the proposed
suggestions, close the bug as duplicate, as invalid or at least inform that
he is aware of the problem and will work on it some time later.

	Just my two cents,

--
[]'s
Fernando Henrique Ferraz P. da Rosa

Bugzilla is a pain-in-the-arse to maintain, unless they've improved it
in the last 9 months.  Just my two cents...

Frank E Harrell Jr writes:

Let me add to the wish list the creation of some mechanism to better track
improvements and bug fixes in packages, such as a change log link by each
package's area in CRAN, or easy access to CVS information from there.
When I report bugs (e.g., in read.xport in foreign [due somewhat to
problems inherent with SAS's format] or ace or avas in acepack) it would
be nice to see some announcement when the bugs are resolved, or to easily
track this.

rossini@blindglobe.net (A.J. Rossini) writes:

Bugzilla is a pain-in-the-arse to maintain, unless they've improved it
in the last 9 months.  Just my two cents...

It might have (and I wouldn't mind replacing Jitterbug with something
that is actually maintained itself!), but there's another rear-end problem... 

Who's going to do the actual work? 

That's my point.  Maintainance is critical, and any time spent on
systems administration (systems in the generic sense, here it's
bug-tracking) is less time spent on other more useful, or interesting,
or high-payoff items.

I'm probably not the only one who wished Peter had more time for other
things...

A.J. Rossini writes:

That's my point.  Maintainance is critical, and any time spent on
systems administration (systems in the generic sense, here it's
bug-tracking) is less time spent on other more useful, or interesting,
or high-payoff items.

I'm probably not the only one who wished Peter had more time for other
things...

        I agree that it would  probably take some time to set up the new system,
and maintaince at the beggining would take time too. On the other hand I believe
that the  producitivy gain fostered by the new system on the long term  could perhaps
counterbalance the initial effort. The main effort would be setting up the new
system and handdling accounts to the package mantainers, after that, most of it
would be left to the users of the system. I don't have much experience with
administrating the Bugzilla system yet but I'm installing in my box and will post
my experiences to the list.

I also share your views about namespaces.  These have caused numerous
problems for me.  It would be nice to have more of a mechanism to put
"feelers" out to the R user community when major changes are planned. 

In Hmisc and Design I reference several functions that were not exported
from packages that now use namespaces.  There is an elegant solution with
the package:::function notation, 

but I have been unable to use this
solution because I use one code base for all versions of R and S-Plus. 
This notation generates syntax errors in all but late versions of R.

On Sat, 17 Jan 2004 09:33:10 -0500, you wrote:

I also share your views about namespaces.  These have caused numerous
problems for me.  It would be nice to have more of a mechanism to put
"feelers" out to the R user community when major changes are planned. 

Changes always show up in r-devel (the main CVS branch, not the
mailing list) first.  Package developers should be keeping a
relatively up to date copy of it around if they're doing things that
are likely to break.

In Hmisc and Design I reference several functions that were not
exported from packages that now use namespaces.  There is an elegant
solution with the package:::function notation, 

I'd recommend avoiding that as much as you can.  If things aren't
exported from a package, then the package writer is likely to feel
free to change them without warning.  It's much better to convince the
package writer that they missed something in their export list.

but I have been unable to use this
solution because I use one code base for all versions of R and S-Plus. 
This notation generates syntax errors in all but late versions of R.

I think it's reasonable to restrict the availability of updates to
your packages to the currently released R version.  There are reasons
why people might not be up to date (e.g. only doing upgrades at a
specific time of year), but they'll still have access via CRAN to
older versions of your package.

Compatibility with S-PLUS is a lot harder, of course.  

Duncan Murdoch

On Sun, 18 Jan 2004 18:47:52 -0500
Duncan Murdoch <dmurdoch@pair.com> wrote:

On Sat, 17 Jan 2004 09:33:10 -0500, you wrote:

I also share your views about namespaces.  These have caused numerous
problems for me.  It would be nice to have more of a mechanism to put
"feelers" out to the R user community when major changes are planned. 

Changes always show up in r-devel (the main CVS branch, not the
mailing list) first.  Package developers should be keeping a
relatively up to date copy of it around if they're doing things that
are likely to break.

I need to do that more often.  But sometimes it's hard to know what things
I do that are likely to break.  That's where there needs to be some other
mechanism for user communications.

In Hmisc and Design I reference several functions that were not
exported from packages that now use namespaces.  There is an elegant

solution with the package:::function notation, 

I'd recommend avoiding that as much as you can.  If things aren't
exported from a package, then the package writer is likely to feel
free to change them without warning.  It's much better to convince the
package writer that they missed something in their export list.

That's a good solution in general, but I could see legitimate
disagreements about what should be exported, so this will not always solve
the problem.

On Sun, 18 Jan 2004 18:47:52 -0500
Duncan Murdoch <dmurdoch@pair.com> wrote:

Changes always show up in r-devel (the main CVS branch, not the
mailing list) first.  Package developers should be keeping a
relatively up to date copy of it around if they're doing things that
are likely to break.

I need to do that more often.  But sometimes it's hard to know what things
I do that are likely to break.  That's where there needs to be some other
mechanism for user communications.

 but they'll still have access via CRAN to
older versions of your package.

"PaulG" == Paul Gilbert <pgilbert@bank-banque-canada.ca>
    on Mon, 19 Jan 2004 11:18:47 -0500 writes:

On Mon, 19 Jan 2004, Frank E Harrell Jr wrote:

On Sun, 18 Jan 2004 18:47:52 -0500
Duncan Murdoch <dmurdoch@pair.com> wrote:

On Sat, 17 Jan 2004 09:33:10 -0500, you wrote:

I also share your views about namespaces.  These have caused
numerous problems for me.  It would be nice to have more of a
mechanism to put"feelers" out to the R user community when major
changes are planned. 

Changes always show up in r-devel (the main CVS branch, not the
mailing list) first.  Package developers should be keeping a
relatively up to date copy of it around if they're doing things that
are likely to break.

I need to do that more often.  But sometimes it's hard to know what
things I do that are likely to break.  That's where there needs to be
some other mechanism for user communications.

Well, there is a NEWS file that is worth consulting, and we (Kurt in 
particular) run all the CRAN packages after every major change and
daily. See  http://cran.r-project.org/src/contrib/checkSummary.html.  We
do also tend to tell package authors directly if their packages break,
at least if they were previously warning-free.

It seems the sort of thing you do is to call methods directly where you 
could equally well call the generic, since that is what is currently 
failing in Design and Hmisc (if survfit.km is a survfit method).

In Hmisc and Design I reference several functions that were not
exported from packages that now use namespaces.  There is an
elegant

That's not showing up in failures on the tests under R-patched.  Of the
listed dependencies only grid, lattice and survival have namespaces, and
only survival has been added since 1.8.1.  (I suspect the R-patched
tests are against the 1.8.1 versions of the recommended packages, not
the current versions.)

solution with the package:::function notation, 

I'd recommend avoiding that as much as you can.  If things aren't
exported from a package, then the package writer is likely to feel
free to change them without warning.  It's much better to convince
the package writer that they missed something in their export list.

That's a good solution in general, but I could see legitimate
disagreements about what should be exported, so this will not always
solve the problem.

I think it does.  If the package writer wants a function to be private,
would-be users should respect that decision.  Most of the cases we have
encountered have been calling methods directly rather than coercing
objects to the right class and calling the generic.  In extremis, copy
(with permission) the function you want from the package sources and
rename it.

Unless I made a mistake there are no current uses of ::: in CRAN
packages, and there are very few in base R (and quite a lot of the
methods::: should probably better be methods::).

Brian

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Duncan Murdoch wrote:

 but they'll still have access via CRAN to
older versions of your package.

Are you sure about that? I can't find old contributed packages, but it 
wouldn't be the first time I've missed something obvious. 
(src/contrib/Old is hardly a complete archive.)

It seems the sort of thing you do is to call methods directly where you
could equally well call the generic, since that is what is currently
failing in Design and Hmisc (if survfit.km is a survfit method).

On Mon, 19 Jan 2004 14:09:58 +0000 (GMT)
Prof Brian Ripley <ripley@stats.ox.ac.uk> wrote:

On Mon, 19 Jan 2004, Frank E Harrell Jr wrote:

On Sun, 18 Jan 2004 18:47:52 -0500
Duncan Murdoch <dmurdoch@pair.com> wrote:

On Sat, 17 Jan 2004 09:33:10 -0500, you wrote:

I also share your views about namespaces.  These have caused
numerous problems for me.  It would be nice to have more of a
mechanism to put"feelers" out to the R user community when major
changes are planned. 

Changes always show up in r-devel (the main CVS branch, not the
mailing list) first.  Package developers should be keeping a
relatively up to date copy of it around if they're doing things that
are likely to break.

I need to do that more often.  But sometimes it's hard to know what
things I do that are likely to break.  That's where there needs to be
some other mechanism for user communications.

Well, there is a NEWS file that is worth consulting, and we (Kurt in 
particular) run all the CRAN packages after every major change and
daily. See  http://cran.r-project.org/src/contrib/checkSummary.html.  We
do also tend to tell package authors directly if their packages break,
at least if they were previously warning-free.

I will start checking NEWS.  The kind of news I need though is more about
bugs that do not cause the package to break.

It seems the sort of thing you do is to call methods directly where you 
could equally well call the generic, since that is what is currently 
failing in Design and Hmisc (if survfit.km is a survfit method).

The point of calling methods directly is efficiency, otherwise I would not
use this dirty practice.  When bootstrapping or otherwise calling methods
repeatedly, I seek the lowest level functions for speed.  This conflicts
with the namespace idea.  I think this should have been taken into
consideration when designing namespaces.

In Hmisc and Design I reference several functions that were not
exported from packages that now use namespaces.  There is an
elegant

That's not showing up in failures on the tests under R-patched.  Of the
listed dependencies only grid, lattice and survival have namespaces, and
only survival has been added since 1.8.1.  (I suspect the R-patched
tests are against the 1.8.1 versions of the recommended packages, not
the current versions.)

Yes, those are the ones.

solution with the package:::function notation, 

I'd recommend avoiding that as much as you can.  If things aren't
exported from a package, then the package writer is likely to feel
free to change them without warning.  It's much better to convince
the package writer that they missed something in their export list.

Right

That's a good solution in general, but I could see legitimate
disagreements about what should be exported, so this will not always
solve the problem.

I think it does.  If the package writer wants a function to be private,
would-be users should respect that decision.  Most of the cases we have
encountered have been calling methods directly rather than coercing
objects to the right class and calling the generic.  In extremis, copy
(with permission) the function you want from the package sources and
rename it.

That is a possibility.  None of the approaches we've named are without
maintenance problems.

Are you sure there is a measurable difference in calling methods
directly? The dispatch overhead on formula (one of your uses) appears to
be about 10 microseconds.  (Note, negligible even for 10,000
bootstraps.)

I believe we took the real performance penalties into account (and 
namespaces had performance pluses as well as minuses).

On Mon, 19 Jan 2004, Prof Brian Ripley wrote:

It seems the sort of thing you do is to call methods directly where
you could equally well call the generic, since that is what is
currently failing in Design and Hmisc (if survfit.km is a survfit
method).

Technically survfit.km() isn't a survival method, since it's called
directly by survfit() rather than by UseMethod, but it works like one.
It's an internal function that has never been documented, which is why
it wasn't exported.

If people want it, I can export it.  I hadn't heard from anyone wanting
it exported.

On Mon, 19 Jan 2004 17:17:39 +0000 (GMT)
Prof Brian Ripley <ripley@stats.ox.ac.uk> wrote:

Are you sure there is a measurable difference in calling methods
directly? The dispatch overhead on formula (one of your uses) appears to
be about 10 microseconds.  (Note, negligible even for 10,000
bootstraps.)

I believe we took the real performance penalties into account (and 
namespaces had performance pluses as well as minuses).

Brian,

I don't worry about dispatch overhead.  I do worry about overhead of
assembling model matrices, removing rows with NAs, etc.  -Frank

The point of calling methods directly is efficiency, otherwise I would not
use this dirty practice.  When bootstrapping or otherwise calling methods

On Mon, 19 Jan 2004, Frank E Harrell Jr wrote:

On Mon, 19 Jan 2004 17:17:39 +0000 (GMT)
Prof Brian Ripley <ripley@stats.ox.ac.uk> wrote:

Are you sure there is a measurable difference in calling methods
directly? The dispatch overhead on formula (one of your uses)
appears to be about 10 microseconds.  (Note, negligible even for
10,000 bootstraps.)

I believe we took the real performance penalties into account (and 
namespaces had performance pluses as well as minuses).

Brian,

I don't worry about dispatch overhead.  I do worry about overhead of
assembling model matrices, removing rows with NAs, etc.  -Frank

Here is what you said:

The point of calling methods directly is efficiency, otherwise I would
not use this dirty practice.  When bootstrapping or otherwise calling
methods

and your code is failing in R-devel because you are calling
formula.default. So, *why* are you calling formula.default?

Calling e.g. glm.fit not glm is not to do with methods, and apparently
survfit.km is not a method.

It might have (and I wouldn't mind replacing Jitterbug with something
that is actually maintained itself!),