Skip to content

Empirical Bayes and mixed effects modeling - are these the same thing?

2 messages · Joshua Rosenberg, Poe, John

Original
Joshua Rosenberg
# 
Hi all, I have been curious about the similarities and differences between
Empirical Bayes and mixed effects modeling approaches. The Wikipedia page
<https://en.wikipedia.org/wiki/Empirical_Bayes_method> for Empirical Bayes,
for instance, says

"Empirical Bayes methods are procedures for statistical inference in which
the prior distribution is estimated from the data. This approach stands in
contrast to standard Bayesian methods, for which the prior distribution is
fixed before any data are observed. Despite this difference in perspective,
empirical Bayes may be viewed as an approximation to a fully Bayesian
treatment of a hierarchical model wherein the parameters at the highest
level of the hierarchy are set to their most likely values, instead of
being integrated out."

This sounds a lot like a mixed effects model, wherein the grand mean /
variance for the outcome represents the prior for the random effects
predictions. Are these the same thing? Just a curiosity and I had trouble
finding helpful answers after look elsewhere.

Josh
Poe, John
# 
They are very related and, depending on how you were trained to view the
material, mixed effects models are a subset of EB. Random effects are
usually (almost always) estimated with shrinkage which is directly related
to EB. In a lot of the software the connection is made very explicit when
estimating random effects in nonlinear models because they are called
empirical bayes means or emperical bayes modes instead of BLUP's (which
don't exist in that context).

Richard McElreath does a great job in his book/lectures making these points
in terms of bayesian shrinkage but the logic holds here just the same.

https://youtu.be/yakg94HyWdE
On Feb 3, 2018 9:27 AM, "Joshua Rosenberg" <jrosen at msu.edu> wrote:
Hi all, I have been curious about the similarities and differences between
Empirical Bayes and mixed effects modeling approaches. The Wikipedia page
<https://en.wikipedia.org/wiki/Empirical_Bayes_method> for Empirical Bayes,
for instance, says

"Empirical Bayes methods are procedures for statistical inference in which
the prior distribution is estimated from the data. This approach stands in
contrast to standard Bayesian methods, for which the prior distribution is
fixed before any data are observed. Despite this difference in perspective,
empirical Bayes may be viewed as an approximation to a fully Bayesian
treatment of a hierarchical model wherein the parameters at the highest
level of the hierarchy are set to their most likely values, instead of
being integrated out."

This sounds a lot like a mixed effects model, wherein the grand mean /
variance for the outcome represents the prior for the random effects
predictions. Are these the same thing? Just a curiosity and I had trouble
finding helpful answers after look elsewhere.

Josh
--
Joshua Rosenberg, Ph.D. Candidate
Educational Psychology ?&? Educational Technology
Michigan State University
http://jmichaelrosenberg.com


_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models