inverse distance weights matrix
Thanks Roger,
I've been working on creating inverse distance matrices; and have been
successful in this first step. However, as you suggested, the
connectedness of these matrices will cause procedures to be slow, even
prohibitive with my computer (P4 3Gz 1GB RAM) (ran out of RAM).
I've tried a simpler first order queen weights matrix to see if I could
get the errorsarlm model to run; but come to a different problem and an
error message:
source("spat.cot.R")
Error in solve.default(asyvar, tol = tol.solve) :
system is computationally singular: reciprocal condition number
= 1.16119e-019
traceback()
6: solve.default(asyvar, tol = tol.solve)
5: solve(asyvar, tol = tol.solve)
4: errorsarlm(YLD02 ~ CLAY + DIST + PEG + ROT + SUN, data = cotton,
q1)
3: eval.with.vis(expr, envir, enclos)
2: eval.with.vis(ei, envir)
1: source("spat.cot.R")
where "spat.cot.R" contains:
spatcot.err<-errorsarlm(YLD02~CLAY+DIST+PEG+ROT+SUN,data=cotton, q1)
Do I need to set my tol to a different value or am I having other
issues?
Thank you,
Terry
-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
Sent: Saturday, March 18, 2006 3:07 PM
To: Griffin, Terry W
Cc: r-sig-geo at stat.math.ethz.ch
Subject: RE: [R-sig-Geo] inverse distance weights matrix
On Sat, 18 Mar 2006, Griffin, Terry W wrote:
Thank you Roger, I've posted the original shape file for the dataset on the website at: http://web.ics.purdue.edu/~twgriffi/R_matrix.htm One shape file is projected while the other is not; however both sets
of
*.dbf are identical and contain both decimal degrees and Northing and Easting in meters. I chose a <=75 meter distance threshold and a power=1. The final inverse distance matrix was row standardized so it is not symmetric. The weight characteristics you list below are nearly identical to those that SpaceStat provides.
OK - it looks like this at the moment:
library(rgdal)
cotton_proj <- readOGR(".", "cotton_proj") # for shapefiles in the
current
# working directory, first argument
plot(cotton_proj, cex=0.2, axes=TRUE)
library(spdep)
d75 <- dnearneigh(coordinates(cotton_proj), 0, 75)
dlist <- nbdists(d75, coordinates(cotton_proj))
idlist <- lapply(dlist, function(x) 1/x)
w75 <- nb2listw(d75, glist=idlist, style="W")
Characteristics of d75:
d75
Neighbour list object: Number of regions: 2451 Number of nonzero links: 1182994 Percentage nonzero weights: 19.69228 Average number of links: 482.6577 But:
inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, ncol=2451,
+ byrow=TRUE)
range(rowSums(inMAT))
[1] 0.8159028 1.9115231
d75MAT <- nb2mat(d75, glist=idlist, style="W") range(rowSums(d75MAT))
[1] 1 1
w75 <- nb2listw(d75, glist=idlist, style="W") range(sapply(w75$weights, sum))
[1] 1 1 So something in the construction of your matrix wasn't securing row standardisation. The values do differ, this is the upper left corner:
d75MAT[1:5,1:5]
[,1] [,2] [,3] [,4] [,5] 1 0.00000000 0.06735710 0.03227068 0.02182278 0.01660474 2 0.05975093 0.00000000 0.05493976 0.02863058 0.01954588 3 0.02615210 0.05019078 0.00000000 0.05461937 0.02771729 4 0.01639324 0.02424508 0.05062943 0.00000000 0.05216365 5 0.01172946 0.01556467 0.02416008 0.04905229 0.00000000
inMAT[1:5,1:5]
[,1] [,2] [,3] [,4] [,5] [1,] 0.000000 0.131420 0.061844 0.041726 0.031848 [2,] 0.101440 0.000000 0.090156 0.047180 0.032441 [3,] 0.040305 0.076122 0.000000 0.083545 0.042787 [4,] 0.023708 0.034730 0.072836 0.000000 0.076431 [5,] 0.016296 0.021506 0.033593 0.068830 0.000000
rowSums(inMAT[1:5,])
[1] 1.911523 1.658689 1.533153 1.440908 1.379051
rowSums(d75MAT[1:5,])
1 2 3 4 5 1 1 1 1 1 They are not the same matrices, although the differences are very small - here I've used the shapefile points, which are stored as binary floating point, rather than using the DBF values, which are a possibly rounded character representation, and that might be enough to cause the difference. I think you should be able to generate w75, and that it should do what you need, but I'd be grateful for feedback on how you get on. Best wishes, Roger PS: with numbers of neighbours between 177 and 678 (both in your matrix and in d75), even allowing for the dampening effect of inverse distance, most analyses will be slow. 75m means that many sites are related to many in your case. Doing: symbols(408725, 3660800, circles=75, inches=FALSE, fg="red", add=TRUE) on top of the earlier plot shows your chosen neighbourhood for an arbitrary point.
I really appreciate this. Thank you, Terry -----Original Message----- From: Roger Bivand [mailto:Roger.Bivand at nhh.no] Sent: Friday, March 17, 2006 5:50 PM To: Griffin, Terry W Cc: r-sig-geo at stat.math.ethz.ch Subject: RE: [R-sig-Geo] inverse distance weights matrix Terry: http://web.ics.purdue.edu/~twgriffi/mat4rf and http://web.ics.purdue.edu/~twgriffi/mat4rs throw access errors. Do you have the coordinates and your specifications for the inverse weighted neighbours (distance threshold, power if any)? inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, ncol=2451, byrow=TRUE) does read the matrix - the weights aren't symmetric, are they? From the weights matrix I get: inMATlw <- mat2listw(inMAT) print(is.symmetric.nb(inMATlw$neighbours)) [1] TRUE print(inMATlw$neighbours) Neighbour list object: Number of regions: 2451 Number of nonzero links: 1182982 Percentage nonzero weights: 19.69208 Average number of links: 482.6528 If you could post the coordinates and your specifications, I could see how to reproduce the weights within R if you like. Roger On Fri, 17 Mar 2006, Griffin, Terry W wrote:
The MATLAB routines are the Spatial Econometrics Toolbox format. I tried the read.dat2listw() in R; the code and error message for a
small
matrix example is below; this uses a full matrix from MATLAB
(realizing
that "read.dat2listw" is intended for sparse matrices).
mattry<-read.dat2listw("mat4rf")
Error in "[.default"(sn, , 1) : incorrect number of dimensions
traceback()
5: NextMethod("[")
4: "[.factor"(sn, , 1)
3: sn[, 1]
2: unique(sn[, 1])
1: read.dat2listw("mat4rf")
I'm not able to use dlmwrite in MATLAB to export the sparse matrix
so
I
tried to import the full matrix into R (code is above). The MATLAB
code
I'm using came from this listserve archive. The MATLAB code and
error
message are below. I'm able to use dlmwrite to create the MATLAB
full
matrix (used above in the R code). load testmat.GWT; A = (spconvert(testmat)); %have to change name in
two
locations on this line
n=size(A); nobs=n(1);
% Normalize matrix (adapted form LeSage 1999)
wt = sparse(A);
[i1,j1,s1]=find(wt);
rsum=sum(wt);
for i=1:nobs;
ind=find(i1==i);
s1(ind,1)=s1(ind,1)/rsum(1,i);
end;
[m,n] = size(wt);
W = sparse(i1,j1,s1,m,n);
S = sparse(W);
dlmwrite('mat4rst', S); %give the new sparse weight matrix a
permanent
name for R
??? Error using ==> sprintf
Function is not defined for sparse inputs.
Error in ==> dlmwrite at 172
str = sprintf(format,m(i,:));
If anyone wants to give it a try, I've put the small example
matrices
and the "real" matrix up on my website at: http://web.ics.purdue.edu/~twgriffi/R_matrix.htm My objective is to use R for my spatial data analysis using inverse distance weights matrices. I'm struggling with the inverse distance matrices. I hope to either create these inverse distance matrices
in
SpaceStat then import into R or create the matrices in R. Thank you, Terry -----Original Message----- From: Roger Bivand [mailto:Roger.Bivand at nhh.no] Sent: Friday, March 17, 2006 2:43 PM To: Griffin, Terry W Cc: r-sig-geo at stat.math.ethz.ch Subject: Re: [R-sig-Geo] inverse distance weights matrix On Thu, 16 Mar 2006, Griffin, Terry W wrote:
Greetings, I'm migrating from MATLAB to R and am having trouble converting
inverse
distance weights matrix into a form usable by R. I've created the matrix in SpaceStat (in *.FMT format) and converted to *.GWT
format
and
also have it in MATLAB *.m format.
I wonder whether your Matlab data are in the Spatial Econometrics Toolbox format - if they are, you could try the read.dat2listw() function mentioned on the read.gwt2nb function help page.
I've tried using the read.gwt2nb in R, but without success. There
are
nearly 2,451 observations with an average of 482 linkages. About
20
%
have nonzero weights. Any suggestions are appreciated.
Without details of what you mean by "without success", it's hard to offer any advice. Could you create a small example of the problems you are
seeing, and repost with copies of the commands you give and any
error
messages?
If you are seeing errors ("Error in ..." messages from R), please
also
type traceback() at the prompt and report the output. You could also
put
the uncooperative GWT file on a website for others to try to see
what
is
going on.
Thank you, Terry Terry W. Griffin Graduate Research Assistant Agricultural Economics Purdue University 403 W State St West Lafayette, IN 47907 765-494-4257 http://web.ics.purdue.edu/~twgriffi/ <http://web.ics.purdue.edu/~twgriffi/> [[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no