Skip to content

Is it worth testing for overdispersion?

5 messages · Iker Vaquero Alba, ONKELINX, Thierry, Paul Johnson +1 more

ONKELINX, Thierry
# 
Dear Iker,

You can add an observation  level random effect to the model. If there is overdisperion in your dataset, then this random effect will have a non-zero variance. You can compare models with and without this observation level random effect to test for overdisperion.

Best regards,

Thierry
2 days later
Paul Johnson
#
On Tue, Jun 21, 2011 at 6:53 AM, Iker Vaquero Alba <karraspito at yahoo.es> wrote:
You are confusing 2 things.  Overdispersion can occur, it may be a problem.

But, we believe the "quasi" sorts of models are just wrong, and if you
are going to treat over dispersion, you need to treat it properly,
either by changing to a negative binomial model or by introducing a
"real" mixed effect, not a quasi approximation of a mixed effect.
Overdispersion would naturally be a property of a non-mixed model,and
you can test for it in many ways. Could I suggest to you Simon
Jackman's package, "pscl", which includes some nice tools for that.

PJ
John Maindonald
# 
"But, we believe the "quasi" sorts of models are just wrong,"

No more wrong, necessarily, than any other way of accounting
for the relevant source of variation.  Quasi models are just less 
explicit than other models that are used for similar purposes,
and therefore less tractable, maybe even unusable, in contexts
where it helps to be able to write down the distribution whose
likelihood on is working with.

They are incompletely specified, straightforward for purposes of 
doing certain calculations, but awkward if one wants to specify the
distributional details of the way that the data were generated.

Negative binomial models are not much better in this respect.
To understand how the data might be approximately negative
binomial, one has to think in terms of a mixing or compounding
or waiting time mechanism.  In the first two cases, these are of 
a very particular kind, involving simpler models.  The data do not 
of themselves make it possible to choose between the alternative 
mechanisms.  Look up "negative binomial" on Wikepedia.  
Negative binomial models seem to me a good way of saying that
one has no idea what may be happening!

The choice between a quasi approximation and observational
level random effects (and sure, the latter have the advantage
that the model is fully specified) should really really hinge on
which, in any particular application, provides the better model
for the variance-covariance structure.  In practice, there is rarely
enough data that it is possible to tell the difference.  

It really is a question that "all models are wrong, but some are useful", 
as in the Box & Draper quote.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm
On 24/06/2011, at 4:00 AM, Paul Johnson wrote: