Different Moran's I correlograms using correlog{ncf} and sp.correlogram{spdep}

Dear Roger,

So what you trying to say about correlog{ncf} is that after the bands 
are computed, all the correlations between data points whose distance 
are within the bin are computed. So I suppose it is a binary kind of 
specification, if pairwise distances are in the bin, the correlations 
will be computed and all these correlations are considered equally in 
computing Moran's I.

So should the correlogram obtained by correlog{ncf} same as the one 
obtained by sp.correlogram(spdep), when the neighbourhood upperlimit is 
same as the increment level of the former.

E.g.,

#Correlogram from correlog{ncf}, where slm1600 is the chosen spatial model

correlog.200b <- correlog(dat$x_long, dat$y_lat, residuals(mod1), na.rm=T,
increment=200, resamp=99, latlon=T)

# Plotting the first 50 bins
plot(correlog.200b$correlation[1:50], type="b", pch=16, cex=1.5, lwd=1.5,
col="red", xlab="distance (10^2 km)", ylab="Moran's I", cex.lab=2,
cex.axis=1.5, ylim=c(-1,1)); abline(h=0)

#should be the same as

nb200 <- dnearneigh(as.matrix(dat$x_long, dat$y_lat), 0, 200, longlat=T)
ME200.listw <- nb2listw(nb200, style="B", zero.policy=T)

correl<-sp.correlogram(nb200, residuals(mod1), order = 50, method = "I", style
= "W", randomisation = TRUE, zero.policy = TRUE, spChk=NULL)

plot(correl1600),

But the results of Moran's I are very different.

Here are the results:

For sp.correlogram(),
Spatial correlogram for residuals(mod1)
method: Moran's I
     estimate expectation    variance standard deviate Pr(I) two sided
1   0.06882699 -0.00123153  0.00021800           4.7449       2.086e-06 ***
2   0.03190221 -0.00127877  0.00027178           2.0127       0.0441461 *
3   0.02914748 -0.00133511  0.00028490           1.8059       0.0709262 .
4   0.04458328 -0.00137363  0.00029717           2.6659       0.0076772 **
5   0.05503366 -0.00139665  0.00031539           3.1775       0.0014854 **
6   0.02696152 -0.00143062  0.00032685           1.5704       0.1163121
7   0.06049141 -0.00144718  0.00036181           3.2563       0.0011289 **
8   0.02526273 -0.00146413  0.00037732           1.3759       0.1688468
9   0.02901564 -0.00147493  0.00039208           1.5399       0.1235966
10  0.06624244 -0.00148810  0.00039586           3.4042       0.0006637 ***
11  0.04189714 -0.00151057  0.00041899           2.1206       0.0339535 *
12 -0.00321915 -0.00154560  0.00045316          -0.0786       0.9373374
.........and so on

For correlog():
$correlation (just first 4 bins)
0.381911078  0.356233456  0.322664149  0.304275005

How should we reconcile the differences?

Many thanks,
Xingli




Quoting Roger Bivand <Roger.Bivand at nhh.no>:

On Thu, 29 Jan 2009, Xingli Giam wrote:

Hello List,

I have been bothered by this problem for the entire day, hope someone
can give me a hand here. Thanks in advance for reading! It could be
really simple and basic so hope you guys bear with it.

We often plot Moran's I spatial correlograms that contrast glm residuals
and spatial model residuals (after incorporating spatial autocorrelation
(SAC) via SEVM, autocov reg, etc) over lag distance classes.

A spatial correlogram is not a well-defined concept. One view is that the
inter-pair distances should be binned, and the output computed for the
pairs of observations in each bin - equivalent to defining widening rings
of neighbours for each observation. There are issues that arise here when
some observations have no neighbours in a given bin - should one adjust
the values used to calculate E(I) and var(I)? This is in correlog() in
ncf, and to a certain extent in pgirmess.

Another view is that the first order set of neighbours should be raised to
higher orders irrespective of how the set was defined: i and j are first
order neighbours, i and k are second order neighbours if j and k are first
order neighbours. This isn't quite matrix powering, because if you are
already a first order neighbour, you can't be a second order as well
(discussed in detail in Cliff & Ord 1981, pp. 118-122). See also ?nblag to
see what is going on. correlog() in pgirmess uses nclass.Sturges() to
choose the number of bins, but does not fully protect against bins with
no-neighbour observations.

After deciding the most appropriate neighbourhood distance (1600 km -
mine's a global dataset), I created a list of weights via nb2listw that
is an inverse function of distance (1/distance). I then performed SEVM
using the list of weights, using the brute force ME() algorithm to
extract eigenvectors that reduce SAC.

I plan to report the Moran's I value given from moran.test() on both my
(non- spatial) GLM and Spatial GLM to prove that the spatial model has
reduced autocorrelation.

In addition to that, I plan to show the correlograms of the model
residuals both non-spatial and spatial GLMs.

I would like to ask which function is a better choice for showing the
correlograms. Should I use correlog{ncf} or sp.correlogram{spdep}. From
what I know, calculating Moran's I require definition of a neighbourhood
and a specific weighting scheme. Does anyone know what is the default
neighbourhood and weighting scheme of correlog()? Is it - if I specify
an increment of 100km, then, all points within 0-100, and then 100-200,
etc, are considered neighbours with a binary weighting scheme for each
distance class?

The ncf correlog() uses many nxn matrices, and computes the bands by:

dkl <- ceiling(dmat/increment)

where dmat is a lower triangle distance matrix, increment is the band
width, and dkl has the same form as dmat (it says which distance band the
moran pairs in the lower triangle crossproduct matrix used belong to).

As you can see, the source code is the documentation here.

For sp.correlogram, can I specify the nb object defined by min:0km,
max:100km, and use a lag order of, for example 50, to plot the
correlogram of my spatial model (which has a neighbourhood distance of
1600 km)?

For both functions, it seems that there is no way to incorporate my
inverse- distance weighting scheme that I used in my spatial models. Is
this of practical concern if I intend to plot my correlograms?

Lastly I have a side question regarding correlograms. When we see
distances of, e.g., 100, 200, 300, etc (the increment or lag distance is
100), on the X-axis of a spatial correlogram, does the point at 200
refer to correlations between neighbours of distance 100-200km apart? Or
does it include the first 100km as well (i.e., correlations between
neighbours of distance 0-200km apart)?

In general, the pairs of distances within that band (or lag order). But
check the code to be sure.

Roger

Sorry for the long post. Many thanks for reading, I really appreciate
it. I know some are really basic questions but I can't find any answers
to my questions on google or searching through the list.

Many thanks,
Xingli

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no