Testing a hypothesis that there was a change after a specific, time point

Thu, Mar 7, 2024 4:34 AM

Dear Santosh Srinivas,

Since there are various ways to model transitions, I will link to an 
article that describes several approaches. Perhaps you will find it helpful.
Bliese, P. D., & Lang, J. W. B. (2016). Understanding relative and 
absolute change in discontinuous growth models: Coding alternatives and 
implications for hypothesis testing. Organizational Research Methods, 
19(4), 562?592. https://doi.org/10.1177/1094428116633502

Depending on your preferences and analytic choices, the interpretation 
of the coefficients changes. Some choices pertain to the modeling of the 
trajectory. For example, one could model the trajectory before the 
program announcement, the trajectory after the program announcement, 
non-linear trajectories, and the immediate effect of the program 
announcement. One could also add random effects for the respective 
change variables. Furthermore, one must decide whether the 
post-announcement coefficients are interpreted in absolute or relative 
terms (i.e., relative to the pre-announcement phase).


Best wishes,

Lukasz Stasielowicz
Osnabr?ck University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Lise-Meitner-Stra?e 3
49076 Osnabr?ck (Germany)
Twitter: https://twitter.com/l_stasielowicz
Tel.: +49 541 969-7735



On 20.02.2024 12:00, r-sig-mixed-models-request at r-project.org wrote:
> Send R-sig-mixed-models mailing list submissions to
> 	r-sig-mixed-models at r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> or, via email, send a message with subject or body 'help' to
> 	r-sig-mixed-models-request at r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-mixed-models-owner at r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-mixed-models digest..."
> 
> 
> Today's Topics:
> 
>     1. Testing a hypothesis that there was a change after a specific
>        time point (Santosh Srinivas)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Mon, 19 Feb 2024 16:41:01 +0000
> From: Santosh Srinivas <santosh.b.srinivas at outlook.com>
> To: "r-sig-mixed-models at r-project.org"
> 	<r-sig-mixed-models at r-project.org>
> Subject: [R-sig-ME] Testing a hypothesis that there was a change after
> 	a specific time point
> Message-ID:
> 	<SJ0PR05MB85191EF80DFE626E6900B81CC9512 at SJ0PR05MB8519.namprd05.prod.outlook.com>
> 	
> Content-Type: text/plain; charset="utf-8"
> 
> 
> Hello Members, I am writing to request your advice on how best to do hypothesis testing for our study.
> 
> Our data looks as follows:
> 
>> head(x)
> # A tibble: 6 ? 7
>    user_id  user_male  days log_days bool_program   dv1     dv2
>    <chr>        <int> <int>    <dbl>        <dbl> <dbl>   <dbl>
> 1 IDX_195          1  1581     7.37            1 0.150 0.00590
> 2 IDX_949          1  1338     7.20            1 0.130 0.0348
> 3 IDX_2428         1   577     6.36            0 0.160 0.0438
> 4 IDX_2312         1   424     6.05            0 0.179 0.0364
> 5 IDX_277          1   790     6.67            0 0.419 0.0515
> 6 IDX_1029         1  1489     7.31            1 0.155 0.0219
>>
> 
> 
> Besides the gender of the user, we have data of users on dv1 and dv2 over 6 years, with the days variable ranging from 0 to 2190 (and log_days being its log transformation).
> 
> We would like to test the hypothesis that a program announcement made on the day 850 caused a significant (potentially, gradual) change in dv1 and/or dv2 scores (regardless of the direction of change) for male and/or female users. The bool_program is set to 0 for days < 1118 and 1 otherwise.
> 
> We are wondering what is the best way to conduct this test, given the hierarchical/nested nature of data.
> 
> We have thus far taken the approach of using lmer:
> 
>> m = lmer(
> +   dv1 ~ user_male + bool_program * log_days + (1|user_id),
> +   data = x
> + )
> 
>> summary(m)
> Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
> Formula: dv1 ~ user_male + bool_program * log_days + (1 | user_id)
>     Data: x
> 
> REML criterion at convergence: -103411.7
> 
> Scaled residuals:
>      Min      1Q  Median      3Q     Max
> -4.0430 -0.3797 -0.0443  0.2992 19.5088
> 
> Random effects:
>   Groups   Name        Variance  Std.Dev.
>   user_id  (Intercept) 0.0004456 0.02111
>   Residual             0.0034587 0.05881
> Number of obs: 37137, groups:  user_id, 1012
> 
> Fixed effects:
>                          Estimate Std. Error         df t value Pr(>|t|)
> (Intercept)            1.878e-01  5.318e-03  9.979e+03  35.310  < 2e-16 ***
> user_male              2.018e-02  2.674e-03  7.157e+02   7.546 1.36e-13 ***
> bool_program           1.036e-01  1.588e-02  3.713e+04   6.524 6.93e-11 ***
> log_days              -6.641e-03  7.185e-04  3.713e+04  -9.243  < 2e-16 ***
> bool_program:log_days -1.522e-02  2.177e-03  3.713e+04  -6.991 2.78e-12 ***
> ---
> Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
> 
> Correlation of Fixed Effects:
>              (Intr) usr_ml bl_prg lg_dys
> user_male   -0.471
> bool_progrm -0.238 -0.017
> log_days    -0.872  0.012  0.289
> bl_prgrm:l_  0.268  0.018 -0.998 -0.329
> 
>> interactions::sim_slopes(m, pred = log_days, modx = bool_program, digits = 3)
> JOHNSON-NEYMAN INTERVAL
> 
> When bool_program is OUTSIDE the interval [-0.672, -0.294], the slope of log_days is p < .05.
> 
> Note: The range of observed values of bool_program is [0.000, 1.000]
> 
> SIMPLE SLOPES ANALYSIS
> Slope of log_days when bool_program = 0.000 (0):
>      Est.    S.E.   t val.       p
> -------- ------- -------- -------
>    -0.007   0.001   -9.243   0.000
> 
> Slope of log_days when bool_program = 1.000 (1):
>      Est.    S.E.    t val.       p
> -------- ------- --------- -------
>    -0.022   0.002   -10.631   0.000
> 
> We are not sure if the approach is right and whether we are specifying the days variable appropriately in lmer. We are also not sure if we should be using a more sophisticated change point approach. We came across some Rpackages such as changepoint, segmented, and strucchange. Are they more appropriate than lmer approach we have used?
> 
> Request your advice.
> 
> Thanks and kind regards
> Srinivas
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 	[[alternative HTML version deleted]]
> 
> 
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> R-sig-mixed-models mailing list
> R-sig-mixed-models at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 
> 
> ------------------------------
> 
> End of R-sig-mixed-models Digest, Vol 206, Issue 8
> **************************************************