[R-pkg-devel] R CMD check crash for Ubuntu 14.04 R-3.2.1 only

  * run the file that causes the crash with R CMD check by itself,
        it works, no crash
  * run R CMD check --use-valgrind rcdd, it works, no crash
  * run R CMD check --use-gct rcdd, it works no crash
  * use tools::check_packages_in_dir to check the tarball, it works no
crash
  * if I run R CMD check with R-3.2.1 or R-devel built from source on this
box,
        it works, no crash

So it seems it is impossible to get the crash except under circumstances
that give almost no debugging info.

What do I do now?

If it helps the tarball in question is

http://users.stat.umn.edu/~geyer/rcdd_1.1-9.tar.gz

--
Charles Geyer
Professor, School of Statistics
University of Minnesota
charlie at stat.umn.edu

        [[alternative HTML version deleted]]

On Fri, Jul 3, 2015 at 11:40 PM, Charles Geyer <charlie at stat.umn.edu>
wrote:
[...]

  * run the file that causes the crash with R CMD check by itself,
        it works, no crash
  * run R CMD check --use-valgrind rcdd, it works, no crash
  * run R CMD check --use-gct rcdd, it works no crash
  * use tools::check_packages_in_dir to check the tarball, it works no
crash
  * if I run R CMD check with R-3.2.1 or R-devel built from source on this
box,
        it works, no crash

So it seems it is impossible to get the crash except under circumstances
that give almost no debugging info.

What do I do now?

Idea: look at the actual valgrind output to see if valgrind reports any
errors?

Gabor

If it helps the tarball in question is

http://users.stat.umn.edu/~geyer/rcdd_1.1-9.tar.gz

--
Charles Geyer
Professor, School of Statistics
University of Minnesota
charlie at stat.umn.edu

        [[alternative HTML version deleted]]

Charles,

I took a look based on your tar ball.  When I use the normal 'loud'
setting I
default to for g++ (ie -pedantic -Wall) or clang++ I get more than a page
full of errors.  I just tried it using the UBSAN-via-clang instrumented R I
provide via a docker container (and for which I showed usage during the
'rocker: Docker on R' tutorial preceding useR! -- slides now on my
presentation page).  It also does not segfault, but it shows a test
difference:

  Running ?bar.R?
    Comparing ?bar.Rout? to ?bar.Rout.save? ...8d7
    <
    51,54c50,53
    < [1,]    0    1    0    1    0    0
    < [2,]    0    1    1    0    0    0
    < [3,]    0    1    0    0    1    0
    < [4,]    0    1    0    0    0    1
    ---

    > [1,]    0    1    0    0    0    1
    > [2,]    0    1    0    0    1    0
    > [3,]    0    1    0    1    0    0
    > [4,]    0    1    1    0    0    0

      Running ?bug2.R?
        Comparing ?bug2.Rout? to ?bug2.Rout.save? ...8d7
        <

(You also seem to consistently differ in one newline which is just a
nuisance
and not an error,)

So no free lunch yet.  I fear you may have to minimize the package somewhat
to hopefully arrive at minimally reproducible example.  So far you seem to
have a Heisenbug (https://en.wikipedia.org/wiki/Heisenbug)

Sorry I cannot offer more help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

The diff for bar.Rout is OK.  The solution is the same (the two matrices
have the same rows and so determine the same convex polyhedron).  Why it
picks different orders of rows on different machines, I have no idea.

I also saw the pages of warnings from gcc -W -Wall -Wextra but they are
all about unused variables and unused arguments of functions.  AFAIK these
are not real problems.

I should say that I did not write most of the code.  This package is an R
interface to the computational geometry provided by cddlib, which is not
something linux distributions package.  So in order to avoid hassle, I just
included the cddlib code (which is GPL) in the src directory and added its
author Komei Fukuda to the authors of the package.

AFAIK this code is very high quality from a numerical analysis standpoint,
however much it may fall below the coding standards of most C coders.
Anyway it is the only free software computational geometry code I know
about, so it is the only game in town unless statisticians want to just say
"we cannot deal with problems like that".

This code may actually segfault somewhere, but we haven't seen it in
practice unless this is its fault (rather than the 32 bit R-3.2.1 build for
Ubuntu 14.04).  We still don't have a way to tell which to blame.

I am trying to follow the debugging rules (http://debuggingrules.com/)
the most important here being "divide and conquer".  Until we know where
the crash occurs, we are just clueless.

On Sat, Jul 4, 2015 at 3:35 AM, Dirk Eddelbuettel <edd at debian.org> wrote:

Charles,

I took a look based on your tar ball.  When I use the normal 'loud'
setting I
default to for g++ (ie -pedantic -Wall) or clang++ I get more than a page
full of errors.  I just tried it using the UBSAN-via-clang instrumented R
I
provide via a docker container (and for which I showed usage during the
'rocker: Docker on R' tutorial preceding useR! -- slides now on my
presentation page).  It also does not segfault, but it shows a test
difference:

  Running ?bar.R?
    Comparing ?bar.Rout? to ?bar.Rout.save? ...8d7
    <
    51,54c50,53
    < [1,]    0    1    0    1    0    0
    < [2,]    0    1    1    0    0    0
    < [3,]    0    1    0    0    1    0
    < [4,]    0    1    0    0    0    1
    ---

    > [1,]    0    1    0    0    0    1
    > [2,]    0    1    0    0    1    0
    > [3,]    0    1    0    1    0    0
    > [4,]    0    1    1    0    0    0

      Running ?bug2.R?
        Comparing ?bug2.Rout? to ?bug2.Rout.save? ...8d7
        <

(You also seem to consistently differ in one newline which is just a
nuisance
and not an error,)

So no free lunch yet.  I fear you may have to minimize the package
somewhat
to hopefully arrive at minimally reproducible example.  So far you seem to
have a Heisenbug (https://en.wikipedia.org/wiki/Heisenbug)

Sorry I cannot offer more help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

--
Charles Geyer
Professor, School of Statistics
University of Minnesota
charlie at stat.umn.edu

Agreed on divide-and-conquer. There is no other way. Valgridn et al may
move
some code to registers for subtle changes.

On 4 July 2015 at 10:20, Charles Geyer wrote:
| I should add a more direct question.  When a crash occurs ONLY when
running R
| CMD check, how does one debug that?  Usually one would say that R CMD
check
| --use-valgrind
| is the answer.  But it only segfaults when --use-vagrind is not used.  So
| somehow we have to debug R CMD check itself.  How?

By "mocking" the exact same (compile and run-time) options used by R CMD
check. You can trace that down.  Things depend a little on whether you do
or
do not use user-library-paths for .libPaths() or not, whether you use
per-user compile flags etc pp.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

I have a general question about appropriate design for a fairly
complicated package, as I am redesigning and rebuilding the optimx
package. It will get a new name as it appears that the calls will change
in order to accommodate new functionality. I plan to make it
fairly easy to add new optimization (function minimization) tools.

The query is whether to use an environment to hold all the common
objects (functions, scaled functions, gradients, scaled gradients,
hessians, scaled hessians, parameters, scaled parameters, program
controls, algorithm tuning parameters, constraints, fixed parametes,
some function value history, etc.) The alternative is a set of very
long calls, especially when the function needing a quantity may be a
couple of layers below the routine that creates or sets the values.

The negative side is that the functions are not portable outside the
package, so would not be exported.

Before I dig into the heavy clay of code that this will need, I thought
I should ask if there are issues I might be missing, as the local
environment -- essentially "global" variables within the perimeter of
the package -- would make the code a lot cleaner, and may offer some
useful efficiencies in computation.

On Fri, Jul 10, 2015 at 12:50 PM, ProfJCNash <profjcnash at gmail.com> wrote:

I have a general question about appropriate design for a fairly
complicated package, as I am redesigning and rebuilding the optimx
package. It will get a new name as it appears that the calls will change
in order to accommodate new functionality. I plan to make it
fairly easy to add new optimization (function minimization) tools.

The query is whether to use an environment to hold all the common
objects (functions, scaled functions, gradients, scaled gradients,
hessians, scaled hessians, parameters, scaled parameters, program
controls, algorithm tuning parameters, constraints, fixed parametes,
some function value history, etc.) The alternative is a set of very
long calls, especially when the function needing a quantity may be a
couple of layers below the routine that creates or sets the values.

The negative side is that the functions are not portable outside the
package, so would not be exported.

Before I dig into the heavy clay of code that this will need, I thought
I should ask if there are issues I might be missing, as the local
environment -- essentially "global" variables within the perimeter of
the package -- would make the code a lot cleaner, and may offer some
useful efficiencies in computation.

If you want to store the same set of things for many optimisers, you
might want to consider using reference classes or R6.

Hadley