Predicted probabilites with CIs for multilevel logistic regression with prior weights

However, should I be using the type = "re" parameter?

This depends on whether you would like to have confidence or prediction 
intervals. I'm not sure if there's a clear definition, or a consensus 
on how to obtain prediction intervals. There's a section about this in 
Ben Bolker's GLMM-FAQ:
http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#predictions-andor-confidence-or-prediction-intervals-on-predictions

He states that you want to add the residual variance to compute 
prediction intervals. However, there are also some caveats: 
https://stackoverflow.com/questions/14358811/extract-prediction-band-from-lme-fit

Ben also has a comment in his examples: "## must be adapted for more 
complex models". I'm not sure if this refers only to make sure to 
compute sigma properly, or if you also need to take the random effect 
variances into account. ggeffects currently computes the standard error 
for type = "re" based on the *random effects variances* (see details 
here: https://easystats.github.io/insight/reference/get_variance.html). 
This seems to be even a bit more conservative than just taking the 
residual variance. But again, any comments on this are welcome. I've 
been in email exchange with Russell Lenth, maintainer of the 
emmeans-package, and we were also talking about if it's actually 
straightforward to obtain prediction intervals, or not. In the latest 
emmeans-release, there is a vignette on this topic 
(https://cran.r-project.org/web/packages/emmeans/vignettes/predictions.html).

In short: you don't need to specify type = "re" if you want to get 
predictions from your mixed models. Predictions for both type = "fe" 
and type = "re" are always on a population-level. However, you need 
type = "re" if you want to make predictions at each level of the random 
effects (group factor), see 
https://strengejacke.github.io/ggeffects/articles/randomeffects.html#marginal-effects-for-each-level-of-random-effects (though here you don't get any intervals).

Best
Daniel

-----Urspr?ngliche Nachricht-----
Von: Sam Crawley <sam_crawley at warpmail.net> 
Gesendet: Dienstag, 18. Juni 2019 03:44
An: d.luedecke <d.luedecke at uke.de>; r-sig-mixed-models at r-project.org
Betreff: Re: [R-sig-ME] Predicted probabilites with CIs for multilevel 
logistic regression with prior weights

Hi again Daniel (and list),

Thanks again for the below. I have been using the ggpredict() function, 
and it works well. However, should I be using the type = "re" 
parameter? Or is this only required when attempting to predict values 
for each group? (I have read the ggeffects documentation on this, but 
it's still not entirely clear to me).

When adding type="re", the confidence intervals become very wide, which 
is obviously not ideal.

Thanks,
Sam Crawley.

On Tue, 11 Jun 2019, at 03:30, d.luedecke at uke.de wrote:

Hi Sam,

you could the "ggeffects" package
(https://strengejacke.github.io/ggeffects/), and there is also an example
for a logistic mixed effects model
(https://strengejacke.github.io/ggeffects/articles/practical_logisticmixedmo
del.html), which might help you.

For binomial models, using weights often results in the following warning:
#> non-integer #successes in a binomial glm!

However, CIs for the predicted probabilities can be calculated nevertheless
(at least in my quick example). Note that afaik, mixed models in R do
correctly not account for sampling weights. However, Thomas Lumley, author
of the survey-package, works on a survey-function for mixed models
(https://github.com/tslumley/svylme), probably the GitHub version is quite
stable (haven't tested yet).

An alternative would be the "scale_weights()" function from the
sjstats-package
(https://strengejacke.github.io/sjstats/articles/mixedmodels-statistics.html
#rescale-model-weights-for-complex-samples ), which rescales sampling
weights so they can be used as "weights" for the mixed models function you
have in R (lme4, lme, ...).

Based on that function, I have a small example that demonstrates how to
compute predicted probabilities for mixed models with (sampling) weights
(ignore the warnings, this is just for demonstration purposes):

library(lme4)
library(sjstats) # for scale_weights() and sample data
library(ggeffects) # for ggpredict()

data(nhanes_sample)
set.seed(123)
nhanes_sample$bin <- rbinom(nrow(nhanes_sample), 1, prob = .3)
nhanes_sample <- scale_weights(nhanes_sample, SDMVSTRA, WTINT2YR)

m <- glmer(
  bin ~ factor(RIAGENDR) * age + factor(RIDRETH1) + (1 | SDMVPSU),
  family = binomial(),
  data = nhanes_sample,
  weights = svywght_a
)

ggpredict(m, c("age", "RIAGENDR")) %>% plot()


Best
Daniel

-----Urspr?ngliche Nachricht-----
Von: R-sig-mixed-models <r-sig-mixed-models-bounces at r-project.org> Im
Auftrag von Sam Crawley
Gesendet: Montag, 10. Juni 2019 10:36
An: r-sig-mixed-models at r-project.org
Betreff: [R-sig-ME] Predicted probabilites with CIs for multilevel logistic
regression with prior weights

Hello all,

I am doing a multilevel binomial logistic regression using lme4, and the
survey data I am using requires weights to be used. I would like to
calculate various predicted probabilities with confidence intervals based on
the estimated model. The predict function obviously doesn't give me standard
errors, and the recommended method to get these is to use the bootMer
function.

However, in my case, the weights provided are not integers, and the bootMer
function exits with an error if the weights are not integers (I raised a
GitHub issue about this, and was pointed to this list:
https://github.com/lme4/lme4/issues/524 ).

So my question is, what is the best way to calculate the predicted
probabilities (with confidence intervals) in my case?

I would appreciate any help you can give me, and I'm happy to provide more
details if required.

Thanks,
Sam Crawley.

--