Testing optimization solvers with equality constraints

As someone who works on trying to improve the optimization codes in R,
though mainly in the unconstrained and bounds-constrained area, I think my
experience is more akin to that of HWB. That is, for some problems -- and
the example in question does have a reparametrization that removes the
constraint, so it could be considered artificial -- there seem to be
many more "bad" starting points than "good" ones. Abby's experience may
be very different, which is more or less the way things are in this type
of work. Or things will work for several years with a class of problems,
then the alligator of an unanticipated "bad" result will take a chunk out
of your posterior.

R users often have advanced expertise in different areas, so I suspect
a lot of cases involve using a poor parameterization. To my mind, the
real issue is less "the program should just work" than "the program
should let the user know there may be issues with the proposed solution".
Such diagnostics are very tricky to devise, and even more awkward to
implement in a way that doesn't become intrusive and time-wasting.
For example, in the optimx package, we can compute the KKT conditions
(first and second derivative checks on proposed solutions), but that
can often take many times the computing effort of finding the solution.
And when there are constraints, KKT conditions are less useful, since
they are for optima "not on the constraint".

This is going to be an ongoing challenge, and it affects a lot of areas
of work now that very complicated calculations are being carried out in
the name of AI and ML (either machine learning or maximum likelihood --
your choice). I'll welcome off-list ideas and efforts to devise better
diagnostics for proposed optimization solutions. We'll need them.

Best, John Nash

Testing optimization solvers with equality constraints

Thread (15 messages)