Dear list:
I'm hoping to tap in to the statistical expertise in the group,
especially those familiar with simulation techniques. I'm finalizing
a
study where I obtain standard errors from two sources. The first
source
is a monte carlo simulation and the other source is an analytical
model
I have developed that appears to recover the standard errors from
the
simulation. All analysis are performed in R using MASS, nlme, and
Matrix.
Here is a very brief description. In the monte carlo, I first sample
from a multivariate distribution to create data. The data are
hypothetical student scores on an achievement test over time and the
aim
is to examine what happens to standard errors under certain
psychometric
conditions. The data are then "contaminated" to reflect a certain
psychometric problem that occurs in longitudinal analyses of student
achievement scores.
These data are then analyzed using a linear model to obtain
parameter
estimates. This is replicated 250 times.
For example, the model equation used is
Y_{ti} = \mu + \beta \cdot t + \epsilon_{ti}
So, I obtain 250 estimates of \mu and \beta. I take the standard
deviation of these estimates to get the sampling distribution of the
parameter (standard errors). Next, I take a single data set,
contaminate
the scores, and then use the analytical approach to obtain standard
errors. So, I end up with two sets of standards errors, those
obtained
under simulated conditions and those obtained from the analytical
model.
My question is what are the most acceptable techniques for comparing
the
standard errors in order to say that the analytical approach
actually
"recovers" the monte carlo standard errors? For the most part, the
standard errors appear to be exactly the same, save rounding error.
One idea I am toying with is to average the standard errors of \mu
and
\beta from the simulation and then do a t-test between the two
standard
errors which might be something along these lines
t = (SE_{analytical} - SE_{mc} )/ \bar se
Where \bar se is the average of the standard errors.
But I'm not certain this is correct. Can anyone suggest a more
appropriate method for comparing the results?
Many thanks. I can also send a copy of the paper to anyone who would
like more information or details.
-Harold
[[alternative HTML version deleted]]