procrustes results affected by order of sites in input file

Fri, Dec 3, 2010 1:58 AM

On Thu, 2010-12-02 at 11:19 -0600, Christine Dolph wrote:

Thanks Christy,

Apologies if I sounded off-hand or dismissive yesterday. It was a busy
day, and as your mail lacked a reproducible example nor the code you
ran, I wanted to deal with the low-hanging fruit before spending time
looking at the issue.

I am currently running a simulation on one of the in-built datasets to
see if I can replicate your findings.

Jari has mentioned, privately, that isoMDS() can be order dependent if
there are ties in the dissimilarities. What, if anything, this has to do
with the protest issue you report is not clear, hence the simulation I
am running.

I will report back later when the simulation has finished... which could
be some time if I continued to code bone-headedly this morning!

Finally, thanks for raising this issue with us.

All the best,

G

Unfortunately, using the set.seed() call does not seem to solve my problem.
If I do not use set.seed(), I do indeed get some small differences in
protest() results due to the effect of random starts. But with my sites in a
given order in the input files, and using set.seed(), I get the same results
every time I run protest().

It's only when I change the order of sites in the input files that I get a
big change in results. The differences I see in protest() results after
re-sorting the order of sites in the input file is quite large, larger than
the smaller differences do to random starts I see if I run protest() without
the set.seed. When sites are re-ordered in the input file, the
correlation-like statistic produced by protest() can jump from between 0.28
to 0.52 for a dataset with 38 sites, with a corresponding change in
significance. Without using set.seed the numbers do jump around a bit, but
not to the same extreme degree.

Looking at the NMDS plots to see how ordinations change with the order of
sites in the file, it seems that if I re-order sites, I am simply getting
ordination solutions that are reflections on one another across one of the 3
NMDS axes. These are not real differences in the ordination solutions. Is
there something I am failing to include in the protest() call that should
enable the function to flip one ordination around to maximize it's
similarity to the other ordination (i.e., that should orient the two
ordinations the same way in 3-dimensional space)?

Thanks,

--Christy


On Thu, Dec 2, 2010 at 6:02 AM, Gavin Simpson <gavin.simpson at ucl.ac.uk>wrote:

On Wed, 2010-12-01 at 14:19 -0600, Christine Dolph wrote:

Dear All,

I am using a Procrustes analysis to compare two NMDS ordinations for the
same set of sites. One ordination is based on fish data, the other is

based

on invertebrate data. Ordinations were derived using metaMDS() from the
{vegan} library as follows:

fish.mds<-metaMDS(fish.data, distance="bray", k=3, trymax=100,
wascores=TRUE, trace=TRUE, zero="add")

invert.mds<-metaMDS(bugcal.a, distance="bray", k=3, trymax=100,
wascores=TRUE, trace=TRUE, zero="add"),


where fish.data and invert.data are site-by-abundance matrices for fish

and

invertebrates, respectively.

I have then used protest() to test the significance between the two
ordinations:

Simple things first:

You did have a set.seed() call before the protest() call didn't you? In
fact, I'd probably put set.seed() calls before the two metaMDS() calls
as well.

The ordering should have no impact at all, but the random starts in
metaMDS() and the random permutations in protest() will vary from run to
run unless you set the random seed to some known value.

G

protest.results<-protest(fish.mds, invert.mds, scale=TRUE,

symmetric=TRUE,

permutations=999)

The problem is, the results of the protest analysis seem to be affected

by

the original ordering of sites in the data input files. That is, if I
re-sort one of the input files based on some criteria, I sometimes see a
change in the strength and significance of the concordance results. In an
attempt to figure out what is happening, I have re-ordered each dataset

in a

number of ways, and plotted the resulting ordinations. I have seen that
while the ordering of sites in the input file does not change the spatial
relationship between them (i.e., does not change the fundamental

ordination

solution), it can change how the sites are oriented with respect to the
different NMDS axes. I was of the belief that a difference in orientation
with respect to the NMDS axes should not affect the Procrustes comparison

of

two ordinations, as the protest function should be rotating and

reflecting

one matrix with respect to the other to find the closest match between

them

(i.e., it should be accounting for differences in how the two solutions

are

oriented in space). However, I appear to see different results depending

on

how sites are oriented with respect to each NMDS axis.

When comparing two ordination solutions with Protest, is it necessary to
ensure that the original data input files are ordered in the same way?

Thanks in advance for any insight.

Sincerely,

--
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
 Dr. Gavin Simpson             [t] +44 (0)20 7679 0522
 ECRC, UCL Geography,          [f] +44 (0)20 7679 0565
 Pearson Building,             [e] gavin.simpsonATNOSPAMucl.ac.uk
 Gower Street, London          [w] http://www.ucl.ac.uk/~ucfagls/<http://www.ucl.ac.uk/%7Eucfagls/>
 UK. WC1E 6BT.                 [w] http://www.freshwaters.org.uk
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%

%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
 Dr. Gavin Simpson             [t] +44 (0)20 7679 0522
 ECRC, UCL Geography,          [f] +44 (0)20 7679 0565
 Pearson Building,             [e] gavin.simpsonATNOSPAMucl.ac.uk
 Gower Street, London          [w] http://www.ucl.ac.uk/~ucfagls/
 UK. WC1E 6BT.                 [w] http://www.freshwaters.org.uk
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%

procrustes results affected by order of sites in input file

Thread (5 messages)