Regression test failed when building on an "older" Linux cluster

An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: https://stat.ethz.ch/pipermail/r-devel/attachments/20071018/a11f817c/attachment.pl
We don't offer the option of building R without documentation, so I 
presume you do not have Perl installed.

Without the examples, there is no check of large swathes of code, so 'make 
check' should not succeed (it would give a false sense of security to pass 
most tests by omitting them).  I've altered it so it will always fail 
without Perl.

I don't buy the disc space argument: you don't have to _install_ the 
documentation on each node, and 'make check' runs on the build and not the 
installed copy.  We don't offer the possibility of installing a cut-down 
version, but surely someone with such a specialized need will be able to 
script that?  And if disc space is tight, why have a copy on each node? 
(We don't on our clusters, where space is not tight: it makes maintenance 
of packages so much easier to have a single shared copy.)

We have an aging Linux Cluster here, running Red Hat 7.3. We have some
business reasons for not upgrading the OS version. I don't recall the exact
hardware (dual Pentium III, 4 Gbyte RAM, 1 GHz clock?), but it was pretty
good in 2001 or so.

We recently tried to build and install R, ver 2.6.0, for this cluster.
It built and apparently ran correctly, but it failed "make check".
Earlier versions of R failed "make check" as well, not always for the same
reason.

I just located the failure in the tests.
It is in the file: R-2.6.0/tests/reg-tests-1.R and fails in the code:

## related checks on eff.aovlist
example(eff.aovlist) # helmert contrasts
eff1 <- eff.aovlist(fit)
fit <- aov(Yield ~ A * B * C + Error(Block), data = aovdat)
eff2 <- eff.aovlist(fit)
stopifnot(all.equal(eff1, eff2)) # will have rounding-error differences
options(contrasts = old)
## Were different in earlier versions

The failure occurred in the 'stopifnot()' call, but was preceded by a
warning about a lack of documentation in "example(eff.aovlist)". I looked at
"example" and found that it relies on available documentation to work,
returning this warning when no documentation is found:

   file <- index.search(topic, INDICES, "AnIndex", "R-ex")
   if (file == "") {
       warning(gettextf("no help file found for '%s'", topic),
           domain = NA)
       return(invisible())

Well, our old Linux cluster doesn't support building R documentation; why
would it when we are very limited on disk space for each node? No one can
log in there; it is all run via LSF. The lack of documentation makes
"example()" fail to return anything and the "reg-tests-1.R" requires that a
value for 'fit' be returned in the global environment for comparison. In
short, in the absence of documentation, this regression test must fail. (I
am assuming that my interpretation of the failure is correct; please feel
free to let me know that I am mis-interpreting the cause of failure!).

If I comment out the above code, all the other tests run correctly. Of
course, normally we would build on another, better equipped host, and then
copy R to each host. That is not possible in this case as the glibc is so
old that it is not compatible with the calls generated on newer hosts.

I am surprised that this problem with "example()" doesn't cause other
failures in the "make check". It makes me think that the use of "example()"
here is unusual (or that I am misinterpreting the failure). If that is the
case, then I suggest removing the call to "example" and replacing it with
the code from within that example. I believe that running R on a compute
cluster without documentation building tools is not an unreasonable use
case. (If that is so, why haven't others stumbled onto this problem before
me? I don't know.) Making the regression tests work in the absence of
documentation, along with the rest of R seems reasonable.

We are now using R on the compute cluster. This email's purpose is to
suggest that R may at times be built in an environment without documentation
tools and it would be nice if the regression tests still worked there. I
hope the developers will consider this use case.

Regards,
Mark Dalphin

PS Having written this email, I am consumed by guilt; I haven't properly and
recently checked the "R admin manual" that I should be able to build R
without documentation. Checking for "essential programs for building and
running R" doesn't seem to show that building documentation is required. I
am relieved to report that building documentation, despite being under
"Essential Programs" section is described in a manner suggesting that
documentation is optional.
http://cran.r-project.org/doc/manuals/R-admin.html#Essential-and-useful-othe
r-programs-under-Unix

----------------------
Mark Dalphin
Dept Comp Biol, M/S AW2/D3262
Amgen, Inc.
1201 Amgen Court W
Seattle, WA 98119
Phone: +1-206-265-7951

	[[alternative HTML version deleted]]

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595
Thank you, Prof. Ripley.

In fact, I tend to agree somewhat with your arguments; our system administrators don't, however.

We do have Perl, though looking now I see its version is too old, 5.6.1, not the required > 5.8.0.

Your argument that the examples are essential for testing sways me. If that is how these tests are constructed, then we have failed the tests and can't trust the use of R on that old cluster. I intended my question to imply that we had failed on only one call to example(), hence I concluded (without evidence) that the test functions called example() only once where I had noted the failure. I probably should have grepped for "example(" before concluding that.

I'll see if we can install an upgraded Perl on a shared location and build R against that. I that solves the problem, I'll report back.

Thank you for your time.

Regards,
Mark Dalphin

-----Original Message-----
From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk]
Sent: Wednesday, October 24, 2007 12:37 AM
To: Dalphin, Mark
Cc: R-Devel at stat.math.ethz.ch
Subject: Re: [Rd] Regression test failed when building on an "older" Linux cluster

We don't offer the option of building R without documentation, so I presume you do not have Perl installed.

Without the examples, there is no check of large swathes of code, so 'make check' should not succeed (it would give a false sense of security to pass most tests by omitting them).  I've altered it so it will always fail without Perl.

I don't buy the disc space argument: you don't have to _install_ the documentation on each node, and 'make check' runs on the build and not the installed copy.  We don't offer the possibility of installing a cut-down version, but surely someone with such a specialized need will be able to script that?  And if disc space is tight, why have a copy on each node?
(We don't on our clusters, where space is not tight: it makes maintenance of packages so much easier to have a single shared copy.)

We have an aging Linux Cluster here, running Red Hat 7.3. We have some
business reasons for not upgrading the OS version. I don't recall the
exact hardware (dual Pentium III, 4 Gbyte RAM, 1 GHz clock?), but it
was pretty good in 2001 or so.

We recently tried to build and install R, ver 2.6.0, for this cluster.
It built and apparently ran correctly, but it failed "make check".
Earlier versions of R failed "make check" as well, not always for the
same reason.

I just located the failure in the tests.
It is in the file: R-2.6.0/tests/reg-tests-1.R and fails in the code:

## related checks on eff.aovlist
example(eff.aovlist) # helmert contrasts
eff1 <- eff.aovlist(fit)
fit <- aov(Yield ~ A * B * C + Error(Block), data = aovdat)
eff2 <- eff.aovlist(fit)
stopifnot(all.equal(eff1, eff2)) # will have rounding-error
differences options(contrasts = old) ## Were different in earlier
versions

The failure occurred in the 'stopifnot()' call, but was preceded by a
warning about a lack of documentation in "example(eff.aovlist)". I
looked at "example" and found that it relies on available
documentation to work, returning this warning when no documentation is found:

   file <- index.search(topic, INDICES, "AnIndex", "R-ex")
   if (file == "") {
       warning(gettextf("no help file found for '%s'", topic),
           domain = NA)
       return(invisible())

Well, our old Linux cluster doesn't support building R documentation;
why would it when we are very limited on disk space for each node? No
one can log in there; it is all run via LSF. The lack of documentation
makes "example()" fail to return anything and the "reg-tests-1.R"
requires that a value for 'fit' be returned in the global environment
for comparison. In short, in the absence of documentation, this
regression test must fail. (I am assuming that my interpretation of
the failure is correct; please feel free to let me know that I am mis-interpreting the cause of failure!).

If I comment out the above code, all the other tests run correctly. Of
course, normally we would build on another, better equipped host, and
then copy R to each host. That is not possible in this case as the
glibc is so old that it is not compatible with the calls generated on newer hosts.

I am surprised that this problem with "example()" doesn't cause other
failures in the "make check". It makes me think that the use of "example()"
here is unusual (or that I am misinterpreting the failure). If that is
the case, then I suggest removing the call to "example" and replacing
it with the code from within that example. I believe that running R on
a compute cluster without documentation building tools is not an
unreasonable use case. (If that is so, why haven't others stumbled
onto this problem before me? I don't know.) Making the regression
tests work in the absence of documentation, along with the rest of R seems reasonable.

We are now using R on the compute cluster. This email's purpose is to
suggest that R may at times be built in an environment without
documentation tools and it would be nice if the regression tests still
worked there. I hope the developers will consider this use case.

Regards,
Mark Dalphin

PS Having written this email, I am consumed by guilt; I haven't
properly and recently checked the "R admin manual" that I should be
able to build R without documentation. Checking for "essential
programs for building and running R" doesn't seem to show that
building documentation is required. I am relieved to report that
building documentation, despite being under "Essential Programs"
section is described in a manner suggesting that documentation is optional.
http://cran.r-project.org/doc/manuals/R-admin.html#Essential-and-usefu
l-othe
r-programs-under-Unix

----------------------
Mark Dalphin
Dept Comp Biol, M/S AW2/D3262
Amgen, Inc.
1201 Amgen Court W
Seattle, WA 98119
Phone: +1-206-265-7951

      [[alternative HTML version deleted]]

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

--
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595