Skip to content
Prev 284027 / 398503 Next

Probit regression with limited parameter space

Ben Bolker 
[cc'ing back to r-help again -- I do this so the answers can be
archived and viewed by others]
On 12-02-02 02:41 PM, Sally Luo wrote:
My best guess is that you are running into optimization problems.

The big advantage of glm() is that it uses a special-purpose
optimization method (iteratively reweighted least squares) that is
generally much more robust/reliable than general-purpose nonlinear
optimizers such as nlminb.  If there is indeed a GLM fitting routine
coded in R, somewhere, that someone has adapted to work with box
constraints, it will probably perform better than mle2.

Some general suggestions for troubleshooting this:

 * check the log-likelihoods returned by the two methods.  If they are
very close (say within 0.01 likelihood units), then the issue is that
you just have a very flat goodness-of-fit surface, and the two sets of
coefficients are in practice very similar to each other.

 * if possible, try starting each approach (glm(), mle2()) from the
solution found by the other (it's a little bit of a pain to get the
syntax right here) and see if they get stuck right where they are or
whether they find that one answer or the other is right.

 * if you were using one of the optimizing methods from optim() (rather
than nlminb), e.g. L-BFGS-B, I would suggest you try using parscale to
rescale the parameters to have approximately equal magnitudes near the
solution.  This apparently isn't possible with nlminb, but you could try
optimizer="optim" (the default), method="L-BFGS-B" and see how you do
(although L-BFGS-B is often a bit finicky).  Alternatively, you can try
optimizer="optimx", in which case you have a larger variety of
unconstrained optimizers to choose from (you have to install the optimx
package and take a look at its documentation).  Alternatively, you can
scale your input variables (e.g. use scale() on your input matrix to get
zero-centered, sd 1 variables), although you would then have to adjust
your lower and upper bounds accordingly.

 * it's a bit more work, but you may be able to unpack this a bit and
provide analytical derivatives.  That would help a lot.

  In short: you are entering the quagmire of numerical optimization methods.

   I have learned most of this stuff by trial and error -- can anyone on
the list suggest a good/friendly introduction?  (Press et al Numerical
Recipes; Givens and Hoeting's Computational Statistics book looks good,
although I haven't read it ...)

  Ben Bolker