Spatial Lag and Error Model

I have read the documentation, and have tried both the LU and MC
methods.  I continue to get a similar error.  With a subset of the data
and a simplified model, then it will execute the command.  In my case, I
am using both linear and quadratic terms for some independent variables
to capture curvature in the marginal effects.  These quadratic terms do
enter strongly significantly in the linear model, but seem to cause
problems for errorsarlm() and lagsarlm(), particularly when the full
sample is used.

Thanks again for the comments.

John.

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
Sent: Saturday, March 26, 2011 4:03 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: RE: [R-sig-Geo] Spatial Lag and Error Model

On Fri, 25 Mar 2011, Janmaat, John wrote:

Hello again,

Thanks for the suggestion of updating the packages.  That dealt with

one

issue.  For the rest, here is some of the info you asked for.  For
errorsarlm and lagsarlm I get a memory allocation problem.

sessionInfo()

R version 2.12.1 (2010-12-16)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United
States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United
States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base


other attached packages:
[1] RANN_2.1.2         pgirmess_1.5.1     spdep_0.5-29
coda_0.14-2        deldir_0.0-13      nlme_3.1-98        MASS_7.3-11
Matrix_0.999375-48
[9] boot_1.2-43        spgwr_0.6-10       maptools_0.8-4
lattice_0.19-17    foreign_0.8-42     sp_0.9-79

loaded via a namespace (and not attached):
[1] grid_2.12.1  tools_2.12.1

Here are the error messages when commands run for full sample:

wd.GMerrorsarSum <- GMerrorsar(frmSum,data=wd2,weightings)
wd.GMerrorsarWin <- GMerrorsar(frmWin,data=wd2,weightings)
wd.errorsarSum <- errorsarlm(frmSum,data=wd2,weightings)

Error: cannot allocate vector of size 922.5 Mb

See ?errorrsarlm, method= argument. You should not expect to be able to
use dense matrix methods with as many observations. For k nearest
neighbours, I suggest LU for an exact result, perhaps MC for an
approximation.

wd.errorsarWin <- errorsarlm(frmWin,data=wd2,weightings)

Error: cannot allocate vector of size 922.5 Mb

Ditto.

wd.MESSlagsarSum <- lagmess(frmSum,data=wd2,weightings)
wd.MESSlagsarWin <- lagmess(frmWin,data=wd2,weightings)
wd.lagsarSum <- lagsarlm(frmSum,data=wd2,weightings)

Error: cannot allocate vector of size 922.5 Mb

Ditto - ?lagsarlm

wd.lagsarWin <- lagsarlm(frmWin,data=wd2,weightings)

Error: cannot allocate vector of size 922.5 Mb

Ditto.

You may also see singularities because your RHS variables appear to be
close to colinear.

wd.GMerrorsarSum <- gstsls(frmSum, data=wd2,weightings)
wd.GMerrorsarWin <- gstsls(frmWin, data=wd2,weightings)

If I run the same set of commands on a subset of the data, then the
command output is:

wd.GMerrorsarSum <- GMerrorsar(frmSum,data=wd2,weightings)
wd.GMerrorsarWin <- GMerrorsar(frmWin,data=wd2,weightings)
wd.errorsarSum <- errorsarlm(frmSum,data=wd2,weightings)

Error in solve.default(asyvar, tol = tol.solve) :
 system is computationally singular: reciprocal condition number =
8.6909e-27

wd.errorsarWin <- errorsarlm(frmWin,data=wd2,weightings)

Error in solve.default(asyvar, tol = tol.solve) :
 system is computationally singular: reciprocal condition number =
8.34845e-27

Both of these come when inverting the asymmetric covariance matrix of
the
variables, read about why this happens on the function help pages - it
is
described in details, and may also concern the scaling of your
variables.

wd.MESSlagsarSum <- lagmess(frmSum,data=wd2,weightings)
wd.MESSlagsarWin <- lagmess(frmWin,data=wd2,weightings)
wd.lagsarSum <- lagsarlm(frmSum,data=wd2,weightings)

Error in solve.default(inf, tol = tol.solve) :
 system is computationally singular: reciprocal condition number =
7.82721e-27

wd.lagsarWin <- lagsarlm(frmWin,data=wd2,weightings)

Error in solve.default(inf, tol = tol.solve) :
 system is computationally singular: reciprocal condition number =
7.81985e-27

Ditto.

wd.GMerrorsarSum <- gstsls(frmSum, data=wd2,weightings)
wd.GMerrorsarWin <- gstsls(frmWin, data=wd2,weightings)

Suggestions?

Read the function help pages fully before you use the functions?

Roger

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
Sent: Friday, March 25, 2011 4:12 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] Spatial Lag and Error Model

On Thu, 24 Mar 2011, Janmaat, John wrote:

-----Original Message-----
From: Janmaat, John
Sent: Thursday, March 24, 2011 3:40 PM
To: 'Roger.Bivand at nhh.no'
Subject: RE: [R-sig-Geo] Spatial Lag and Error Model

Hello Roger,

Thanks for your reply.

A code snippet:

nl <- knearneigh(cbind(wd2$Long,wd2$Lat),k=4)
nb <- knn2nb(nl)
weightings <- nb2listw(nb)
wd.GMerrorsarSum <- gstsls(frmWin, data=wd2,weightings)

The error message:

Error in solve.default(QQ, Qye) :
 system is computationally singular: reciprocal condition number =
4.20784e-28

There are 10,996 observations.

You did not provide sessionInfo() output. If you update spdep, you'll
find
that this issue (of poorly conditioned crossproduct matrices in the
stsls
step) was addressed in December 2010, and the current release should
work.

The code and error messages from errorsarlm() and lagsarlm() will
probably
also show other misunderstandings, but that can be addressed if you

post

them.

Hope this helps,

Roger

The dataset has been processed to remove any overlapping points (no

zero

distances) and to remove points that do not have at least one

neighbour

within 100m.  Overlapping points existed as accounts would change

when

a

resident moved, leading to multiple observations for a single lot.

This

is what I originally thought could be the issue, as there are then

zero

distances.  It has also had all observations with NA removed.  The

model

represented in frmWin is solved fine by lm(), with no variables

dropped.

I'm not sure how to check for near linear dependence in WX though, so

I

would appreciate it if you could direct me there.

I am using k nearest neighbours, as opposed to rook or queen as my

data

has lat and long for the lot centroid, as opposed to a polygon for

each

lot.

There are some natural boundaries within the data, such that it can

be

divided into subsets where within each subset the neighbour list for

the

observations in the subset is no different from that generated for

the

whole dataset.  I have also done an analysis separately for such
subsets, with the same result.

Thanks in advance for any suggestions.

John.

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
Sent: Thursday, March 24, 2011 1:27 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] Spatial Lag and Error Model

On Wed, 23 Mar 2011, Janmaat, John wrote:

Hello,



I have a city water use dataset with almost 12000 observations.  I

am

using spdep.  I am using K nearest neighbours (4) to build the

neighbour

list.  I am trying to estimate a model to predict household water

use,

controlling for a set of independent variables like lot size, house
size, etc.  The spatial model diagnostics, lm.LMtests, suggests that
there is both a spatial error and a spatial lag in the data.

However,

on the lm estimations (lagsarlm,errorsarlm), report that the system

is

singular.  I can estimate the model using GMerrorsar and lagmess.
However, gstsls exits with the same error as the lm estimations.

Is there a function I am missing to estimate a model with both

spatial

autocorrelation and a spatial lagged dependent variable that won't

have

the singularity problem?

Well, what we are missing are the verbatim error messages and

function

calls. It may well be that your diagnosis of the problem is not

precise

enough, especially as the code used depends on your choice of input
arguments. Does lm() of the same model report any unfitted

coefficients

(are there near-linear dependencies present in the X variables, or
between
X and WX)?

Roger

Thanks,



John.



--------------------

Dr. John Janmaat

Department of Economics

I.K. Barber School of Arts and Sciences

University of British Columbia - Okanagan Campus

3333 University Way, Kelowna, BC

V1V 1V7

Tel: (250)807-8021

WWW: http://people.ok.ubc.ca/jjanmaat/




	[[alternative HTML version deleted]]