Message-ID: <CANOgrHY_eA8aQvcbucxv+atyQO9LoT8iMA6M0AHiMC_EruuX9A@mail.gmail.com>
Date: 2021-05-10T17:20:27Z
From: Mitchell Maltenfort
Subject: contradictory odds ratios--a problem with the equation or the interpretation?
In-Reply-To: <CAJB+otq5M-F3E_qHa4gQuzF6=TGydAC3EzCGw1BU+e=Y8kwepA@mail.gmail.com>
Look at the confidence intervals. Mandarin and Spanish overlap.
On Mon, May 10, 2021 at 1:16 PM Johnathan Jones <johnathan.jones at gmail.com>
wrote:
> Hi all,
>
> I?m running a generalized linear mixed model in R (4.0.3) and while most
> findings are in-line with what could be expected, I?m getting one that?s
> off. Either I?m mixing something up in my equation or there?s a reasonable
> explanation for my results that I?m not seeing. I'm hoping someone here
> might be able to diagnose the issue.
>
>
> *Background*
>
> I am researching speech perception in second languages (n = 53, 48 items).
> Specifically, I am investigating how well a person's ability to accurately
> perceive words spoken in isolation predicts their ability to perceive words
> spoken in sentences.
>
> Different language groups have different language transfer issues which
> complicate things.
>
> Also impacting perception is association--whether you associate the word
> with its sentence context.
>
>
> *Variables of interest*
>
> Outcome variable (Y): perception of a word in a sentence
>
> Predictor variables:
>
> - iso1: the participant?s estimated ability to identify a word in isolation
> (a performance score. I?ve used raw and Rasch standardised scores here to
> see if results would change. No dice).
>
> - iso2: the participant?s estimated ability to discriminate between
> isolated words in sequences (a performance score, as described in iso1).
>
> - language: what language group the participant is from. Languages include
> English, Mandarin, and Spanish (note: Mandarin consistently outperforms
> Spanish in raw and standard scores]
>
> - association: whether the participant associates the target word in the
> sentence with the sentence. Association levels include same, different, and
> neutral (it's a little more nuanced, but this communicates what's
> necessary).
>
> Random variables: participant and item.
>
> Equation: Y ~ iso1 + iso2 + language + association + (1|participant) +
> (1|item)
>
>
> *Outputs *(via sjPlot)
>
> Predictor Odds ratio CI
>
> (Intercept) 58.45 18.47-184.90
>
>
> - Iso1: 1.02 1.00-1.03
>
> - Iso2: 1.03 1.01-1.04
>
> - Association [same]: 2.44 0.23-0.49
>
> - Association [different]: 0.34 1.64-3.61
>
> - Language [Mandarin]: 0.04 0.01-0.12
>
> - Language [Spanish]: 0.05 0.01-.18
>
>
>
> The good from the output:
>
> Association works out. Participants have greater log odds of obtaining a
> correct answer when they associate the word with its sentential context.
> Not associating the word with the context tends to lead to misperception.
> There is a pretty large effect here.
>
> The bad from the output:
>
> Language is yielding opposite results than expected. The Spanish group has
> an odds ratio of .05 while the Mandarin group has an odds ratio of .04.
> This is irregular as Mandarin outperforms Spanish across all tasks
> (evidenced by raw scores and Rasch analysis).
>
>
> If the equation looks right, how can it be that a lower performing group
> (by every other task or metric) has a better odds ratio than a higher
> performing group when predicting performance?
>
> Any ideas as to what I might try to resolve the language variable issue or
> possible interpretations of what I see as a wonky result would be very much
> appreciated.
>
>
> Thank you!
>
>
> John Jones
> E: johnathan.jones at gmail.com
> SM: linkedin.com/in/johnathanjones
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
--
Sent from Gmail Mobile
[[alternative HTML version deleted]]