glm gives incorrect results for zero-weight cases (PR#780)

ripley@stats.ox.ac.uk writes:

The reason is obvious: glm.fit only ever updates eta[good], and 
zero-weight values are not `good'.  So eta[weights == 0] is stuck at the
initial values.

There are two possible fixes:

1) Update eta after the final fit, and then mu.  Out of range values
could then be NA (although it looks like predict.glm does not check).

2) Update all eta and hence mu values during the iterations.  This will
apply the constraints on eta/mu at zero-weight points too, and so might
be different.

I am inclined to think that 2) is right, and that adding points with zero 
weight to the fit is not the same as omitting them.

Opinions?

Just for clarification: This applies only to cases where the
parametrization is non-canonical, e.g. additive models with Poisson
response, right? And essentially the issue is that if you have a model
like lambda = a + b x and you put in a zero-weight observation with x
= 0, then that should effectively constrain a to be positive. That
does make quite good sense, yes.

The reason is obvious: glm.fit only ever updates eta[good], and 
zero-weight values are not `good'.  So eta[weights == 0] is stuck at the
initial values.

There are two possible fixes:

1) Update eta after the final fit, and then mu.  Out of range values
could then be NA (although it looks like predict.glm does not check).

2) Update all eta and hence mu values during the iterations.  This will
apply the constraints on eta/mu at zero-weight points too, and so might
be different.

I am inclined to think that 2) is right, and that adding points with zero 
weight to the fit is not the same as omitting them.

Opinions?

ripley@stats.ox.ac.uk writes:

The reason is obvious: glm.fit only ever updates eta[good], and 
zero-weight values are not `good'.  So eta[weights == 0] is stuck at the
initial values.

There are two possible fixes:

1) Update eta after the final fit, and then mu.  Out of range values
could then be NA (although it looks like predict.glm does not check).

2) Update all eta and hence mu values during the iterations.  This will
apply the constraints on eta/mu at zero-weight points too, and so might
be different.

I am inclined to think that 2) is right, and that adding points with zero 
weight to the fit is not the same as omitting them.

Opinions?

Just for clarification: This applies only to cases where the
parametrization is non-canonical, e.g. additive models with Poisson
response, right? And essentially the issue is that if you have a model
like lambda = a + b x and you put in a zero-weight observation with x
= 0, then that should effectively constrain a to be positive. That
does make quite good sense, yes.