inverse distance weights matrix

Thanks Roger,

I've been working on creating inverse distance matrices; and have been
successful in this first step.  However, as you suggested, the
connectedness of these matrices will cause procedures to be slow, even
prohibitive with my computer (P4 3Gz 1GB RAM) (ran out of RAM).

I've tried a simpler first order queen weights matrix to see if I 
couldget the errorsarlm model to run; but come to a different 
problem and an
error message:

source("spat.cot.R")
Error in solve.default(asyvar, tol = tol.solve) : 
       system is computationally singular: reciprocal condition 
number= 1.16119e-019

traceback()

6: solve.default(asyvar, tol = tol.solve)
5: solve(asyvar, tol = tol.solve)
4: errorsarlm(YLD02 ~ CLAY + DIST + PEG + ROT + SUN, data = 
cotton, 
      q1)
3: eval.with.vis(expr, envir, enclos)
2: eval.with.vis(ei, envir)
1: source("spat.cot.R")

where "spat.cot.R" contains:
spatcot.err<-errorsarlm(YLD02~CLAY+DIST+PEG+ROT+SUN,data=cotton, q1)

Do I need to set my tol to a different value or am I having other
issues?

Thank you,

Terry
 



-----Original Message-----
From: Roger Bivand [Roger.Bivand at nhh.no] 
Sent: Saturday, March 18, 2006 3:07 PM
To: Griffin, Terry W
Cc: r-sig-geo at stat.math.ethz.ch
Subject: RE: [R-sig-Geo] inverse distance weights matrix

On Sat, 18 Mar 2006, Griffin, Terry W wrote:

Thank you Roger,

I've posted the original shape file for the dataset on the 

website at:

http://web.ics.purdue.edu/~twgriffi/R_matrix.htm

One shape file is projected while the other is not; however both 

setsof

*.dbf are identical and contain both decimal degrees and 

Northing and

Easting in meters.  

I chose a <=75 meter distance threshold and a power=1.  

The final inverse distance matrix was row standardized so it is not
symmetric. The weight characteristics you list below are nearly
identical to those that SpaceStat provides.  

OK - it looks like this at the moment:

library(rgdal)
cotton_proj <- readOGR(".", "cotton_proj") # for shapefiles in the
current
               	# working directory, first argument
plot(cotton_proj, cex=0.2, axes=TRUE)
library(spdep)
d75 <- dnearneigh(coordinates(cotton_proj), 0, 75)
dlist <- nbdists(d75, coordinates(cotton_proj))
idlist <- lapply(dlist, function(x) 1/x)
w75 <- nb2listw(d75, glist=idlist, style="W")

Characteristics of d75:

d75

Neighbour list object:
Number of regions: 2451 
Number of nonzero links: 1182994 
Percentage nonzero weights: 19.69228 
Average number of links: 482.6577 

But:

inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, 

ncol=2451, 
+ byrow=TRUE)

range(rowSums(inMAT))

[1] 0.8159028 1.9115231

d75MAT <- nb2mat(d75, glist=idlist, style="W")
range(rowSums(d75MAT))

[1] 1 1

w75 <- nb2listw(d75, glist=idlist, style="W")
range(sapply(w75$weights, sum))

[1] 1 1

So something in the construction of your matrix wasn't securing 
row 
standardisation. The values do differ, this is the upper left corner:

d75MAT[1:5,1:5]

       [,1]       [,2]       [,3]       [,4]       [,5]
1 0.00000000 0.06735710 0.03227068 0.02182278 0.01660474
2 0.05975093 0.00000000 0.05493976 0.02863058 0.01954588
3 0.02615210 0.05019078 0.00000000 0.05461937 0.02771729
4 0.01639324 0.02424508 0.05062943 0.00000000 0.05216365
5 0.01172946 0.01556467 0.02416008 0.04905229 0.00000000

inMAT[1:5,1:5]

        [,1]     [,2]     [,3]     [,4]     [,5]
[1,] 0.000000 0.131420 0.061844 0.041726 0.031848
[2,] 0.101440 0.000000 0.090156 0.047180 0.032441
[3,] 0.040305 0.076122 0.000000 0.083545 0.042787
[4,] 0.023708 0.034730 0.072836 0.000000 0.076431
[5,] 0.016296 0.021506 0.033593 0.068830 0.000000

rowSums(inMAT[1:5,])

[1] 1.911523 1.658689 1.533153 1.440908 1.379051

rowSums(d75MAT[1:5,])

1 2 3 4 5 
1 1 1 1 1 

They are not the same matrices, although the differences are very 
small- 
here I've used the shapefile points, which are stored as binary 
floating
point, rather than using the DBF values, which are a possibly 
rounded 
character representation, and that might be enough to cause the 
difference. I think you should be able to generate w75, and that it
should 
do what you need, but I'd be grateful for feedback on how you get on.

Best wishes,

Roger

PS: with numbers of neighbours between 177 and 678 (both in your 
matrix 
and in d75), even allowing for the dampening effect of inverse 
distance,
most analyses will be slow. 75m means that many sites are related to
many 
in your case. Doing:

symbols(408725, 3660800, circles=75, inches=FALSE, fg="red", add=TRUE)

on top of the earlier plot shows your chosen neighbourhood for an 
arbitrary point.

I really appreciate this.  Thank you,

Terry



-----Original Message-----
From: Roger Bivand [Roger.Bivand at nhh.no] 
Sent: Friday, March 17, 2006 5:50 PM
To: Griffin, Terry W
Cc: r-sig-geo at stat.math.ethz.ch
Subject: RE: [R-sig-Geo] inverse distance weights matrix

Terry:

http://web.ics.purdue.edu/~twgriffi/mat4rf

and 

http://web.ics.purdue.edu/~twgriffi/mat4rs

throw access errors.

Do you have the coordinates and your specifications for the 

inverse 

weighted neighbours (distance threshold, power if any)? 

inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, 

ncol=2451, 

  byrow=TRUE)

does read the matrix - the weights aren't symmetric, are they?

From the weights matrix I get:

inMATlw <- mat2listw(inMAT)

print(is.symmetric.nb(inMATlw$neighbours))
[1] TRUE

print(inMATlw$neighbours)
Neighbour list object:
Number of regions: 2451 
Number of nonzero links: 1182982 
Percentage nonzero weights: 19.69208 
Average number of links: 482.6528 

If you could post the coordinates and your specifications, I 

could see

how 
to reproduce the weights within R if you like.

Roger


On Fri, 17 Mar 2006, Griffin, Terry W wrote:

The MATLAB routines are the Spatial Econometrics Toolbox 

format.  I

tried the read.dat2listw() in R; the code and error message 

for a

small

matrix example is below; this uses a full matrix from MATLAB

(realizing

that "read.dat2listw" is intended for sparse matrices). 

mattry<-read.dat2listw("mat4rf")

Error in "[.default"(sn, , 1) : incorrect number of dimensions

traceback()

5: NextMethod("[")
4: "[.factor"(sn, , 1)
3: sn[, 1]
2: unique(sn[, 1])
1: read.dat2listw("mat4rf")


I'm not able to use dlmwrite in MATLAB to export the sparse matrix

so

I

tried to import the full matrix into R (code is above).  The 

MATLAB> code

I'm using came from this listserve archive.  The MATLAB code and

error

message are below.  I'm able to use dlmwrite to create the MATLAB

full

matrix (used above in the R code).

load testmat.GWT; A = (spconvert(testmat)); %have to change 

name in

two

locations on this line
n=size(A); nobs=n(1);
 % Normalize matrix (adapted form LeSage 1999)
wt = sparse(A);
 [i1,j1,s1]=find(wt);
rsum=sum(wt);
for i=1:nobs;
            ind=find(i1==i);
s1(ind,1)=s1(ind,1)/rsum(1,i);
end;
[m,n] = size(wt);
W = sparse(i1,j1,s1,m,n);
S = sparse(W);
dlmwrite('mat4rst', S); %give the new sparse weight matrix a

permanent

name for R 
??? Error using ==> sprintf
Function is not defined for sparse inputs.

Error in ==> dlmwrite at 172
        str = sprintf(format,m(i,:));

If anyone wants to give it a try, I've put the small example

matrices

and the "real" matrix up on my website at:
http://web.ics.purdue.edu/~twgriffi/R_matrix.htm

My objective is to use R for my spatial data analysis using 

inverse> > distance weights matrices.  I'm struggling with the 
inverse distance

matrices.  I hope to either create these inverse distance matrices

in

SpaceStat then import into R or create the matrices in R.  

Thank you,

Terry




-----Original Message-----
From: Roger Bivand [Roger.Bivand at nhh.no] 
Sent: Friday, March 17, 2006 2:43 PM
To: Griffin, Terry W
Cc: r-sig-geo at stat.math.ethz.ch
Subject: Re: [R-sig-Geo] inverse distance weights matrix

On Thu, 16 Mar 2006, Griffin, Terry W wrote:

Greetings,

 

I'm migrating from MATLAB to R and am having trouble converting

inverse

distance weights matrix into a form usable by R.  I've 

created the

matrix in SpaceStat (in *.FMT format) and converted to *.GWT

format

and

also have it in MATLAB *.m format.

I wonder whether your Matlab data are in the Spatial Econometrics
Toolbox 
format - if they are, you could try the read.dat2listw() 

function 

mentioned on the read.gwt2nb function help page.

 

I've tried using the read.gwt2nb in R, but without success.  

There> are

nearly 2,451 observations with an average of 482 linkages.  

About20

%

have nonzero weights.  Any suggestions are appreciated.

Without details of what you mean by "without success", it's 

hard to

offer 
any advice. Could you create a small example of the problems 

you are

seeing, and repost with copies of the commands you give and any

error 

messages? 

If you are seeing errors ("Error in ..." messages from R), please

also

type traceback() at the prompt and report the output. You 

could also

put

the uncooperative GWT file on a website for others to try to see

what

is

going on.

 

Thank you,

Terry

 

 

Terry W. Griffin

Graduate Research Assistant

Agricultural Economics

Purdue University

403 W State St

West Lafayette, IN 47907

765-494-4257

http://web.ics.purdue.edu/~twgriffi/
<http://web.ics.purdue.edu/~twgriffi/> 

 

 


	[[alternative HTML version deleted]]

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian 
School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no