[EXTERNAL] Re: survival changes

On 6/1/19 1:32 PM, Marc Schwartz wrote:

On Jun 1, 2019, at 12:59 PM, Peter Langfelder <

peter.langfelder at gmail.com> wrote:

On Sat, Jun 1, 2019 at 3:22 AM Therneau, Terry M., Ph.D. via R-devel
<r-devel at r-project.org> wrote:

In the next version of the survival package I intend to make a

non-upwardly compatable

change to the survfit object.  With over 600 dependent packages this

is not something to

take lightly, and I am currently undecided about the best way to go

about it.  I'm looking

for advice.

The change: 20+ years ago I had decided not to include the initial

x=0,y=1 data point in

the survfit object itself.  It was not formally an estimand and the

plot/points/lines etc

routines could add this on themselves.  That turns out to have been a

mistake, and has led

to a steady proliferation of extra bits as I realized that the time

axis doesn't always

start at 0, and later (with multi state) that y does not always start

at 1 (though the

states sum to 1), and later the the error doesn't always start at 0,

and another

realization with cumulative hazard, and ...
The new survfit method for multi-state coxph models was going to add

yet another special

case.  Basically every component is turning into a duplicate of "row

1" vs "all the

others".  (And inconsistently named.)

Three possible solutions
1. Current working draft of survival_3.0.3:  Add a 'version' element

to the survfit object

and a 'survfit2.3' function that converts old to new.  All my

downstream functions (print,

plot,...) start with an "if (old) update to new" line.  This has

allowed me to stage

updates to the functions that create survfit objects -- I expect it to

happen slowly.

There will also be a survfit3.2 function to go backwards. Both the

forward and backwards

functions leave objects alone if they are currently in the desired

format.

2. Make a new class "survfit3" and the necessary 'as' functions. The

package would contain

plot.survfit and plot.survfit3 methods, the former a two line "convert

and call the

second" function.

3. Something I haven't thought of.

A more "clean break" solution would be to start a whole new package
(call it survival2) that would make these changes, and deprecate the
current survival. You could add warnings about deprecation and urging
users to switch in existing survival functions. You could continue
bugfixes for survival but only add new features to survival2. The new
survival2 and the current survival could live side by side on CRAN for
quite some time, giving maintainers of dependent packages (and just
plain users) enough time to switch. This could allow you to
change/clean up other parts of the package that you could perhaps also
use a rethink/rewrite, without too much concern for backward
compatibility.

Peter

Hi,

I would be cautious in going in that direction, bearing in mind that

survival is a Recommended package, therefore included in the default R
distribution from the R Foundation and other parties. To have two versions
can/will result in substantial confusion, and I would argue against that
approach.

There is language in the CRAN submission policy that covers API changes,

which strictly speaking, may or may not be the case here, depending upon
which direction Terry elects to go:

"If an update will change the package?s API and hence affect packages

depending on it, it is expected that you will contact the maintainers of
affected packages and suggest changes, and give them time (at least 2
weeks, ideally more) to prepare updates before submitting your updated
package. Do mention in the submission email which packages are affected and
that their maintainers have been informed. In order to derive the reverse
dependencies of a package including the addresses of maintainers who have
to be notified upon changes, the function
reverse_dependencies_with_maintainers is available from the developer
website."

Given the potential extent and impact of the changes being considered,

it would seem reasonable to:

1. Post a note to R-Devel (possibly R-Help to cover a larger useR base)

regarding whatever changes are finalized and formally announce them. The
changes are likely to affect end useRs as well as package maintainers.

2. Send communications directly via e-mail to the relevant package

maintainers that have dependencies on survival.

3. Consider a longer deprecation time frame for relevant functions, to

raise awareness and allow for changes to be made by package maintainers and
useRs as may be apropos. Perhaps post reminders to R-Help at relevant time
points in advance as you approach the formal deprecation and release of the
updated package.

Terry, if you have not used it yet and/or are not aware of it, take a

look at ?Deprecated in base:

https://stat.ethz.ch/R-manual/R-devel/library/base/html/Deprecated.html

which is helpful in setting up a deprecation process. If you Google

"deprecating functions in R", there are numerous examples/flows of use and
the associated processes, since the help page does not contain any examples
at present.

Regards,

Marc Schwartz

Thanks for the comments thus far.
I think that a new package is out of the question, survival is just too
deeply embedded.

Note that my standard policy is to run R CMD check on ALL of the packages
in CRAN that
depend, import, or suggest survival, before any update to CRAN.  Well,
almost all: I skip
the bioconductor ones, and there are a handful that want a GIS or some
other addition not
on my test box.  It usually takes a week or more to chase down
everything.  Then the CRAN
folks do the same, and often find one or two that I missed.  I plan to
make the github
version available well before pushing to CRAN.  Any change to the package
will be painful
to me as well.

Terry T.