LRT tests in lmer

Dear Ben/Rob.

  

As far as I can tell, the standard advice is simply to look at the predictions of the model, compare them with the data, and try to spot any systematic patterns in the residuals.
    

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in its  prediction ( Jarrord Hadfield). And this is reflected in the residual plot with the the long lines of equal residuals all belonging  to the same family - i.e 200 - 600 is the orchid family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

  

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the predictions of the model, compare them with the data, and try to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in its  prediction ( Jarrord Hadfield). And this is reflected in the residual plot with the the long lines of equal residuals all belonging  to the same family - i.e 200 - 600 is the orchid family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the  
predictions of the model, compare them with the data, and try to  
spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in  
its  prediction ( Jarrord Hadfield). And this is reflected in the  
residual plot with the the long lines of equal residuals all  
belonging  to the same family - i.e 200 - 600 is the orchid family  
and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

  If you want to do population-level predictions from a GLMM (i.e.  
setting all random effects to zero), the basic recipe is to (1)  
construct a model (design) matrix for the desired sets of predictor  
variables (if you want to the predict the observed data rather than  
some other set, you can just extract the model matrix from the  
fitted object); (2) multiply it by the vector of fixed effect  
coefficients; (3) transform it back to the scale of the observations  
with the inverse link function.  There's an example on p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf 
 ...

Thats great thanks,

But will this work where you have a binary response variable or will  
the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the  
predictions of the model, compare them with the data, and try to  
spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in  
its  prediction ( Jarrord Hadfield). And this is reflected in the  
residual plot with the the long lines of equal residuals all  
belonging  to the same family - i.e 200 - 600 is the orchid family  
and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

 If you want to do population-level predictions from a GLMM (i.e.  
setting all random effects to zero), the basic recipe is to (1)  
construct a model (design) matrix for the desired sets of predictor  
variables (if you want to the predict the observed data rather than  
some other set, you can just extract the model matrix from the  
fitted object); (2) multiply it by the vector of fixed effect  
coefficients; (3) transform it back to the scale of the observations  
with the inverse link function.  There's an example on p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf 
 ...

Thats great thanks,

But will this work where you have a binary response variable or will the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the predictions of the model, compare them with the data, and try to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in its  prediction ( Jarrord Hadfield). And this is reflected in the residual plot with the the long lines of equal residuals all belonging  to the same family - i.e 200 - 600 is the orchid family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

If you want to do population-level predictions from a GLMM (i.e. setting all random effects to zero), the basic recipe is to (1) construct a model (design) matrix for the desired sets of predictor variables (if you want to the predict the observed data rather than some other set, you can just extract the model matrix from the fitted object); (2) multiply it by the vector of fixed effect coefficients; (3) transform it back to the scale of the observations with the inverse link function.  There's an example on p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf ...

Hi Jarrord,

I have tried using MCMCglmm, however the posterior distributions of  
the majority of the fixed factors straddle 0, which i have read is a  
problem, likely with the priors.

HPDintervals - https://files.me.com/chrismcowen/wqq1lu

prior=list(R=list(V=1, fix=1), G=list(G1=list(V=1, nu=0),  
G2=list(V=1, nu=0)))

So i am unsure how to interpret the results, as to ascertain the  
importance of each factor.

Unfortunately i don't know enough about baysian statistics or R to  
alter my model so the interpretations become clearer.

An example

                             			lower      		upper
(Intercept)             			-3.510792767 	2.40740650
STOStorage organ        	-0.299408836 	0.23073133
BSUnisexual flower      	-0.131660436 	0.54887912
BSUnisexual plant       	 0.003566637 	0.81742862
PDBiotic                			 0.054625970 	0.72436838
PDMammalia              		-2.139720264 	1.39753939



On 11 Aug 2010, at 16:37, Jarrod Hadfield wrote:

Hi Chris,

It is hard to say as it will depend on the fixed effects. In  
addition its not clear whether such a situation is diagnostic of a  
problem.  Imagine you just have an intercept which is estimated to  
be exactly zero. The residuals on the data scale will be either 0.5  
or -0.5, but this does not imply the model is wrong.

Cheers,

Jarrod

On 11 Aug 2010, at 15:41, Chris Mcowen wrote:

Thats great thanks,

But will this work where you have a binary response variable or  
will the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at  
the predictions of the model, compare them with the data, and try  
to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in  
its  prediction ( Jarrord Hadfield). And this is reflected in the  
residual plot with the the long lines of equal residuals all  
belonging  to the same family - i.e 200 - 600 is the orchid family  
and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

If you want to do population-level predictions from a GLMM (i.e.  
setting all random effects to zero), the basic recipe is to (1)  
construct a model (design) matrix for the desired sets of predictor  
variables (if you want to the predict the observed data rather than  
some other set, you can just extract the model matrix from the  
fitted object); (2) multiply it by the vector of fixed effect  
coefficients; (3) transform it back to the scale of the  
observations with the inverse link function.  There's an example on  
p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf ...

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Hi Jarrord,

I have tried using MCMCglmm, however the posterior distributions of the majority of the fixed factors straddle 0, which i have read is a problem, likely with the priors.

HPDintervals - https://files.me.com/chrismcowen/wqq1lu

prior=list(R=list(V=1, fix=1), G=list(G1=list(V=1, nu=0), G2=list(V=1, nu=0)))

So i am unsure how to interpret the results, as to ascertain the importance of each factor.

Unfortunately i don't know enough about baysian statistics or R to alter my model so the interpretations become clearer.

An example

                            			lower      		upper
(Intercept)             			-3.510792767 	2.40740650
STOStorage organ        	-0.299408836 	0.23073133
BSUnisexual flower      	-0.131660436 	0.54887912
BSUnisexual plant       	 0.003566637 	0.81742862
PDBiotic                			 0.054625970 	0.72436838
PDMammalia              		-2.139720264 	1.39753939



On 11 Aug 2010, at 16:37, Jarrod Hadfield wrote:

Hi Chris,

It is hard to say as it will depend on the fixed effects. In addition its not clear whether such a situation is diagnostic of a problem.  Imagine you just have an intercept which is estimated to be exactly zero. The residuals on the data scale will be either 0.5 or -0.5, but this does not imply the model is wrong.

Cheers,

Jarrod

On 11 Aug 2010, at 15:41, Chris Mcowen wrote:

Thats great thanks,

But will this work where you have a binary response variable or will the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the predictions of the model, compare them with the data, and try to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in its  prediction ( Jarrord Hadfield). And this is reflected in the residual plot with the the long lines of equal residuals all belonging  to the same family - i.e 200 - 600 is the orchid family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

If you want to do population-level predictions from a GLMM (i.e. setting all random effects to zero), the basic recipe is to (1) construct a model (design) matrix for the desired sets of predictor variables (if you want to the predict the observed data rather than some other set, you can just extract the model matrix from the fitted object); (2) multiply it by the vector of fixed effect coefficients; (3) transform it back to the scale of the observations with the inverse link function.  There's an example on p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf ...

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Sorry about the formatting,

i was not going to use P values for model selection, rather the DIC  
value

Iterations = 12991
Thinning interval  = 3001
Sample size  = 1000

DIC: 3171.501

G-structure:  ~order

     post.mean  l-95% CI u-95% CI eff.samp
order      7720 4.023e-13  0.09208     1000

              ~fam:fam

       post.mean  l-95% CI u-95% CI eff.samp
fam:fam   4092456 2.376e-12  0.02938     1000

R-structure:  ~units

     post.mean l-95% CI u-95% CI eff.samp
units         1        1        1        0

Location effects: IUCN ~ STO + BS + PD + FR + END + WO + RG + SEA +  
ALT + BIO + SE + FS

                        			post.mean   l-95% CI   u-95% 		CI  
eff.samp pMCMC
(Intercept)              		39.065870  -3.510793   2.407406   1000.0  
		0.776
STOStorage organ        	 -0.004916  -0.299409   0.230731    757.2		  
0.946
BSUnisexual flower       	 0.211852  -0.131660   0.548879    708.0 		 
0.212
BSUnisexual plant         	0.370895   0.003567   0.817429    770.3 		 
0.070 .
PDBiotic                  		0.381261   0.054626   0.724368    774.4  
		0.040 *
PDMammalia               		26.364377  -2.139720   1.397539   1000		. 
0 0.724
FRNon_fleshy_fruit       	-0.208198  -0.536699   0.083012    964.2 		 
0.202
ENDNon_endospermous   0.503829   0.200868   0.822120    591.7 		 
0.004 **
WOWoody                  		-0.203632  -0.565069   0.139240    857.5  
		0.272
RGTwo+                   		-0.052508  -0.250675   0.163811    831.8  
		0.588
SEAHapaxanthic           	-1.344993  -4.504625   1.848373    890.4 		 
0.406
SEAHapaxanthic          	  0.223060  -1.590483   2.012970    785.9 		 
0.800
SEAPerennial             		-0.097971  -0.460607   0.304681    849.9  
		0.580
SEAPleonanthic       	       -0.069756  -0.813837   0.704066     
969.4 		0.872
ALTHigh                 		 -0.129331  -0.483238   0.200436   1000.0  
		0.472
ALTLow                  		 -0.171467  -0.514753   0.121200    842.9  
		0.316
ALTMid                   		 0.068307  -0.227978   0.379701    814.9  
		0.660
BIOBoreal                 		1.785916  -1.222387   4.769563    860.2  
		0.254
BIOMediterranean-type     2.105530  -0.888236   4.786029    817.9 		 
0.156
BIOSubantarctic           	2.214561  -0.888921   5.239470    841.3 		 
0.190
BIOSubarctic            		  2.441894  -0.667793   5.677992    849.5  
		0.142
BIOSubtropical/Tropical     2.336425  -0.660675   4.899198    928.3  
		0.124
BIOTemperate             		 2.315834  -0.761101   4.826330    809.2  
		0.132
SEFew-Several           		146.220538  -0.620787   3.933475   1000.0  
		0.172
SENumerous              	  	0.206148  -0.117869   0.572987    734.9  
		0.236
SESeveral                 		0.626675  -0.236956   1.456895    881.7  
		0.134
SESingle                		  0.399690   0.030041   0.779923    709.8  
		0.032 *
FSZygomorphic            	 0.032334  -0.215194   0.265597    355.7 		 
0.814
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

Cutpoints:
                    post.mean l-95% CI u-95% CI eff.samp
cutpoint.traitIUCN.1    0.6593   0.5211    0.793    48.46
cutpoint.traitIUCN.2    2.4694   2.2952    2.663    41.37
cutpoint.traitIUCN.3    3.6258   3.4220    3.827    38.02
cutpoint.traitIUCN.4    4.1156   3.9166    4.341    52.46
On 11 Aug 2010, at 17:15, Jarrod Hadfield wrote:

Hi,

Could you give summary(model) with the new version (2.05) - it will  
be easier to see what is going on?

Jarrod
On 11 Aug 2010, at 17:08, Chris Mcowen wrote:

Hi Jarrord,

I have tried using MCMCglmm, however the posterior distributions of  
the majority of the fixed factors straddle 0, which i have read is  
a problem, likely with the priors.

HPDintervals - https://files.me.com/chrismcowen/wqq1lu

prior=list(R=list(V=1, fix=1), G=list(G1=list(V=1, nu=0),  
G2=list(V=1, nu=0)))

So i am unsure how to interpret the results, as to ascertain the  
importance of each factor.

Unfortunately i don't know enough about baysian statistics or R to  
alter my model so the interpretations become clearer.

An example

                           			lower      		upper
(Intercept)             			-3.510792767 	2.40740650
STOStorage organ        	-0.299408836 	0.23073133
BSUnisexual flower      	-0.131660436 	0.54887912
BSUnisexual plant       	 0.003566637 	0.81742862
PDBiotic                			 0.054625970 	0.72436838
PDMammalia              		-2.139720264 	1.39753939



On 11 Aug 2010, at 16:37, Jarrod Hadfield wrote:

Hi Chris,

It is hard to say as it will depend on the fixed effects. In  
addition its not clear whether such a situation is diagnostic of a  
problem.  Imagine you just have an intercept which is estimated to  
be exactly zero. The residuals on the data scale will be either 0.5  
or -0.5, but this does not imply the model is wrong.

Cheers,

Jarrod

On 11 Aug 2010, at 15:41, Chris Mcowen wrote:

Thats great thanks,

But will this work where you have a binary response variable or  
will the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at  
the predictions of the model, compare them with the data, and  
try to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in  
its  prediction ( Jarrord Hadfield). And this is reflected in the  
residual plot with the the long lines of equal residuals all  
belonging  to the same family - i.e 200 - 600 is the orchid  
family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

If you want to do population-level predictions from a GLMM (i.e.  
setting all random effects to zero), the basic recipe is to (1)  
construct a model (design) matrix for the desired sets of  
predictor variables (if you want to the predict the observed data  
rather than some other set, you can just extract the model matrix  
from the fitted object); (2) multiply it by the vector of fixed  
effect coefficients; (3) transform it back to the scale of the  
observations with the inverse link function.  There's an example  
on p. 6 of http://glmm.wdfiles.com/local--files/examples/ 
Owls.pdf ...

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

are all 5's for example associated with a single fixed factor, or something like this?

G1=list(V=1, nu=1, alpha.mu=0, alpha.V=1000)

Sorry about the formatting,

i was not going to use P values for model selection, rather the DIC value

Iterations = 12991
Thinning interval  = 3001
Sample size  = 1000

DIC: 3171.501

G-structure:  ~order

   post.mean  l-95% CI u-95% CI eff.samp
order      7720 4.023e-13  0.09208     1000

            ~fam:fam

     post.mean  l-95% CI u-95% CI eff.samp
fam:fam   4092456 2.376e-12  0.02938     1000

R-structure:  ~units

   post.mean l-95% CI u-95% CI eff.samp
units         1        1        1        0

Location effects: IUCN ~ STO + BS + PD + FR + END + WO + RG + SEA + ALT + BIO + SE + FS

                      			post.mean   l-95% CI   u-95% 		CI eff.samp pMCMC
(Intercept)              		39.065870  -3.510793   2.407406   1000.0 		0.776
STOStorage organ        	 -0.004916  -0.299409   0.230731    757.2		 0.946
BSUnisexual flower       	 0.211852  -0.131660   0.548879    708.0 		0.212
BSUnisexual plant         	0.370895   0.003567   0.817429    770.3 		0.070 .
PDBiotic                  		0.381261   0.054626   0.724368    774.4 		0.040 *
PDMammalia               		26.364377  -2.139720   1.397539   1000		.0 0.724
FRNon_fleshy_fruit       	-0.208198  -0.536699   0.083012    964.2 		0.202
ENDNon_endospermous   0.503829   0.200868   0.822120    591.7 		0.004 **
WOWoody                  		-0.203632  -0.565069   0.139240    857.5 		0.272
RGTwo+                   		-0.052508  -0.250675   0.163811    831.8 		0.588
SEAHapaxanthic           	-1.344993  -4.504625   1.848373    890.4 		0.406
SEAHapaxanthic          	  0.223060  -1.590483   2.012970    785.9 		0.800
SEAPerennial             		-0.097971  -0.460607   0.304681    849.9 		0.580
SEAPleonanthic       	       -0.069756  -0.813837   0.704066    969.4 		0.872
ALTHigh                 		 -0.129331  -0.483238   0.200436   1000.0 		0.472
ALTLow                  		 -0.171467  -0.514753   0.121200    842.9 		0.316
ALTMid                   		 0.068307  -0.227978   0.379701    814.9 		0.660
BIOBoreal                 		1.785916  -1.222387   4.769563    860.2 		0.254
BIOMediterranean-type     2.105530  -0.888236   4.786029    817.9 		0.156
BIOSubantarctic           	2.214561  -0.888921   5.239470    841.3 		0.190
BIOSubarctic            		  2.441894  -0.667793   5.677992    849.5 		0.142
BIOSubtropical/Tropical     2.336425  -0.660675   4.899198    928.3 		0.124
BIOTemperate             		 2.315834  -0.761101   4.826330    809.2 		0.132
SEFew-Several           		146.220538  -0.620787   3.933475   1000.0 		0.172
SENumerous              	  	0.206148  -0.117869   0.572987    734.9 		0.236
SESeveral                 		0.626675  -0.236956   1.456895    881.7 		0.134
SESingle                		  0.399690   0.030041   0.779923    709.8 		0.032 *
FSZygomorphic            	 0.032334  -0.215194   0.265597    355.7 		0.814
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

Cutpoints:
                  post.mean l-95% CI u-95% CI eff.samp
cutpoint.traitIUCN.1    0.6593   0.5211    0.793    48.46
cutpoint.traitIUCN.2    2.4694   2.2952    2.663    41.37
cutpoint.traitIUCN.3    3.6258   3.4220    3.827    38.02
cutpoint.traitIUCN.4    4.1156   3.9166    4.341    52.46
On 11 Aug 2010, at 17:15, Jarrod Hadfield wrote:

Hi,

Could you give summary(model) with the new version (2.05) - it will be easier to see what is going on?

Jarrod
On 11 Aug 2010, at 17:08, Chris Mcowen wrote:

Hi Jarrord,

I have tried using MCMCglmm, however the posterior distributions of the majority of the fixed factors straddle 0, which i have read is a problem, likely with the priors.

HPDintervals - https://files.me.com/chrismcowen/wqq1lu

prior=list(R=list(V=1, fix=1), G=list(G1=list(V=1, nu=0), G2=list(V=1, nu=0)))

So i am unsure how to interpret the results, as to ascertain the importance of each factor.

Unfortunately i don't know enough about baysian statistics or R to alter my model so the interpretations become clearer.

An example

                         			lower      		upper
(Intercept)             			-3.510792767 	2.40740650
STOStorage organ        	-0.299408836 	0.23073133
BSUnisexual flower      	-0.131660436 	0.54887912
BSUnisexual plant       	 0.003566637 	0.81742862
PDBiotic                			 0.054625970 	0.72436838
PDMammalia              		-2.139720264 	1.39753939



On 11 Aug 2010, at 16:37, Jarrod Hadfield wrote:

Hi Chris,

It is hard to say as it will depend on the fixed effects. In addition its not clear whether such a situation is diagnostic of a problem.  Imagine you just have an intercept which is estimated to be exactly zero. The residuals on the data scale will be either 0.5 or -0.5, but this does not imply the model is wrong.

Cheers,

Jarrod

On 11 Aug 2010, at 15:41, Chris Mcowen wrote:

Thats great thanks,

But will this work where you have a binary response variable or will the residuals clump around 1 and 0?

Chris
On 11 Aug 2010, at 15:31, Ben Bolker wrote:

On 10-08-11 10:21 AM, Chris Mcowen wrote:

Dear Ben/Rob.

As far as I can tell, the standard advice is simply to look at the predictions of the model, compare them with the data, and try to spot any systematic patterns in the residuals.

I have plotted the residuals of my model - https://files.me.com/chrismcowen/v586vx

I have been made aware that  that lmer uses the random effects in its  prediction ( Jarrord Hadfield). And this is reflected in the residual plot with the the long lines of equal residuals all belonging  to the same family - i.e 200 - 600 is the orchid family and 650-100 is the grass family.

So is there a work around with a glmm?



Thanks

Chris

If you want to do population-level predictions from a GLMM (i.e. setting all random effects to zero), the basic recipe is to (1) construct a model (design) matrix for the desired sets of predictor variables (if you want to the predict the observed data rather than some other set, you can just extract the model matrix from the fitted object); (2) multiply it by the vector of fixed effect coefficients; (3) transform it back to the scale of the observations with the inverse link function.  There's an example on p. 6 of http://glmm.wdfiles.com/local--files/examples/Owls.pdf ...

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.