Controlling for self-selection bias / endogeneity in mixed models

Hi Kelly,

It sounds like you've got correct reasoning on the need for a
multilevel model if your variable of interest is time invariant.

Can you post a link to the thread you're referencing?

A bit of clarity on the flavor(s) of endogeneity that concern you
might be helpful. The omitted variable bias issues solved by group
mean centering and the Mundlak device are mostly from model
mis/underspecification whereas sample selection is a fundamentally
different mechanism. Both are common sources of endogeneity recognized
as such in different pockets of econ but they tend to be seen as
fundamentally different (often conceptually
unrelated) problems in other fields. Econ subsumes omitted variables,
joint causation, measurement error, and sample selection under the
endogeneity umbrella because they all cause correlation between X and
the error but other fields don't make the same connection. For
instance, early panel data work talked about Mundlak devices as
"instruments" in the same way that dynamic panel data models talk
about lags and first differences as instruments but they aren't
traditional instrumental variables that you'd find in the wild and
arguably wouldn't pass the exclusion restriction test outside of panel
data. They call them instruments because they instrument the
endogeneity but they aren't "instrumental variables" in the common parlance.

It's not clear to me if you are referring to general omitted variable
bias whereby you don't have all the appropriate variables in the model
or sample selection bias a la Heckman whereby the sample under study
is systematically different from the population to which you would
like to make inferences and thus needs some kind of complex propensity
to choose A or B style correction like with the standard selection
model. I'm not clear specifically because you referenced the inverse
mills ratio but it *sounds* like you just think you are possibly
missing some set of confounders due to the lack of randomization. If
you do have sample selection bias you can use a multilevel variant of
a heckman selection model with random effects in the outcome and selection equations. See Grilli, L., & Rampichini, C.
(2010). Selection bias in linear mixed models. *Metron, 68*(3),
309-329 for the best discussion of the topic that I've read. Most
multilevel modeling work with this kind of problem is based on
multilevel propensity score matching which is a close cousin of
multilevel Heckman selection models as the inverse mills ratio and the propensity score are related.

You're right that the addition of group means per Mundlak segregates
the within and between effects into two different sets of betas when
they would otherwise be a weighted average. It's just a
reparamaritization of the dummy variable version of fixed effects. It
is mathematically impossible in a linear model for a group mean
centered multilevel model to return different within group beta
coefficients than the standard FE model. That doesn't mean that both
of them aren't wrong because of cross-level interactions, measurement
error, selection bias and what not but they would both be wrong in
identical ways. You can directly test that they are identical with a
version of a Hausman test comparing the within group betas with a chi2
test. The degrees of freedom calculation will be off from the regular
test because the between effects add extra but the within effects will
be identical to rounding error so it really won't matter. You can also
just do a Mundlak variation on the test. All panel data econometrics
textbooks outline this and you can justify the modeling strategy that way regardless of reviewer misconceptions.

If the FE or group mean centered MLM are both wrong and there's some
kind of interactive effect still at work then a random coefficient
will likely show up as mattering for model fit with something like an
LR test. If beta
(X_i-Xbar_j) on Y does not vary as a function of group per an LR test
or something fancier like WAIC then it is reasonable (but not
infallible) evidence that you don't have group heterogeneity-related
omitted variable bias which is what economists would typically be
concerned about in this context. You can still have other kinds of
bias at work just like with any other kind of observational model. The
random coefficient in this context is a regularized interactive fixed
effect in econ jargon whereby you are interacting the grouping
structure with whatever X you want and getting a distribution of
effects. Fundamentally, it's like saying you have some kind of
conditional relationship between group/person and X and just
interacting them. It's slightly complicated by the fact that empirical bayes shrinkage exists but if you have balanced panels then it's mostly a non issue.



On Sun, Apr 12, 2020 at 7:34 PM Slaughter, Kelly
<KELLY.SLAUGHTER at tcu.edu<mailto:KELLY.SLAUGHTER at tcu.edu>>
wrote:

Hi all -

I have a concern regarding self-selection/omitted variable bias. I
have a longitudinal/repeated measures model, theorizing about a
relationship between treatment/control and effort, represented in nlme syntax as:

EQ 1) log(effort measured in time) ~ treatment*scale(experience),
random = ~1|subject

Treatment/control is selected by the subject, it is not randomized,
thus raising endogeneity concerns. My background is applied econ, so
as I learn the mixed model domain, I expected to find the mixed
model equivalent of instrumental variables/inverse Mills ratio, etc.
Yet there is surprisingly (to me) limited material addressing this
issue. The best reference material I found is in fact a thread in
this mailing list from October 2016 and the papers referenced within, leading to Bell, Fairbrother, and Jones (2019).
My first impression is that I should employ a within-between random
effects (REWB)model -

EQ 2) log(effort measured in time) ~ treatment*scale(experience) +
experience_between + experience_within, random = experience_within +
scale(experience) | subject

If I understand correctly, the intuition is that the addition of a
group mean explanatory variable "breaks out" the variability that
would be associated with an omitted variable / error term. Per Bell
et al, "there can be no correlation between level 1 variables
included in the model and the level 2 random effects...unchanging
and/or unmeasured characteristics of an individual (such as
intelligence, ability, etc.) will be controlled out of the estimate of the within effect."

So, no concern between the subject (level 2) and treatment (level 1)
via REWB, wonderful!

Bell et al caution, "...in a REWB/Mundlak models, unmeasured level 2
characteristics can cause bias in the estimates of between effects
and effects of other level 2 variables."

Not an issue for me - I am not concerned with level 2, I include
subject to address the IID violation but am interested in
population, not subject, performance.

Bell et al continue, "However, unobserved time-varying
characteristics can still cause biases at level 1 in either an FE or a REWB/Mundlak model."

Though conceptually my treatment variable is time-varying (it can
change across time within a subject), as a practical/empirical
matter, the treatment is unchanging within the subject - subjects
have no reason to change / would prefer to keep the choice constant.
Of 80k records, treatment switches within a subject occur in about a dozen records.

So, I think I have my solution. However, if a reviewer is not happy
with the with-in / between REWB solution (worried about the level 1
bias), I can further defend EQ 2 via its random coefficient/slope,
if I understand the Oct 2016 thread correctly.

So, my questions are:

(1) Is the above correctly reasoned?

(2) If the random slope model is a further defense against
self-selection bias, could someone provide an intuitive explanation
as to why? Is the idea that by allowing slopes to vary, there is no
endogeneity problem to solve as the very structure of the model
makes the correlated errors concern irrelevant?

Other solutions I explore include a Mundlak model, but per Bell et
al, the Mundlak models are not meaningful for repeated measures.
Also, it appears that the brms package appears to support mixed
modeling using instrumental variables, something I am more
comfortable with per my background, but strong instrumental variables are hard to find in the wild!

Thank you! - Kelly


        [[alternative HTML version deleted]]

        [[alternative HTML version deleted]]