Skip to content

Spatial Lag and Error Model

6 messages · Janmaat, John, Roger Bivand

Janmaat, John
# 
-----Original Message-----
From: Janmaat, John 
Sent: Thursday, March 24, 2011 3:40 PM
To: 'Roger.Bivand at nhh.no'
Subject: RE: [R-sig-Geo] Spatial Lag and Error Model

Hello Roger,

Thanks for your reply.

A code snippet:

nl <- knearneigh(cbind(wd2$Long,wd2$Lat),k=4)
nb <- knn2nb(nl)
weightings <- nb2listw(nb)
wd.GMerrorsarSum <- gstsls(frmWin, data=wd2,weightings)

The error message:

Error in solve.default(QQ, Qye) : 
  system is computationally singular: reciprocal condition number =
4.20784e-28

There are 10,996 observations.

The dataset has been processed to remove any overlapping points (no zero
distances) and to remove points that do not have at least one neighbour
within 100m.  Overlapping points existed as accounts would change when a
resident moved, leading to multiple observations for a single lot.  This
is what I originally thought could be the issue, as there are then zero
distances.  It has also had all observations with NA removed.  The model
represented in frmWin is solved fine by lm(), with no variables dropped.

I'm not sure how to check for near linear dependence in WX though, so I
would appreciate it if you could direct me there.

I am using k nearest neighbours, as opposed to rook or queen as my data
has lat and long for the lot centroid, as opposed to a polygon for each
lot.

There are some natural boundaries within the data, such that it can be
divided into subsets where within each subset the neighbour list for the
observations in the subset is no different from that generated for the
whole dataset.  I have also done an analysis separately for such
subsets, with the same result.

Thanks in advance for any suggestions.

John.

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no] 
Sent: Thursday, March 24, 2011 1:27 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] Spatial Lag and Error Model
On Wed, 23 Mar 2011, Janmaat, John wrote:

            

      
      
      
    

        
neighbour

            

      
      
      
    

        
have

            

      
      
      
    

        
Well, what we are missing are the verbatim error messages and function 
calls. It may well be that your diagnosis of the problem is not precise 
enough, especially as the code used depends on your choice of input 
arguments. Does lm() of the same model report any unfitted coefficients 
(are there near-linear dependencies present in the X variables, or
between 
X and WX)?

Roger

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Thu, 24 Mar 2011, Janmaat, John wrote:

        

            

      
      
      
    

        
You did not provide sessionInfo() output. If you update spdep, you'll find 
that this issue (of poorly conditioned crossproduct matrices in the stsls 
step) was addressed in December 2010, and the current release should work.

The code and error messages from errorsarlm() and lagsarlm() will probably 
also show other misunderstandings, but that can be addressed if you post 
them.

Hope this helps,

Roger

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Janmaat, John
    

    

      
      #
    

  


  

      
Hello again,

Thanks for the suggestion of updating the packages.  That dealt with one
issue.  For the rest, here is some of the info you asked for.  For
errorsarlm and lagsarlm I get a memory allocation problem.

            

      
      
      
    

        
R version 2.12.1 (2010-12-16)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United
States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United
States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base


other attached packages:
 [1] RANN_2.1.2         pgirmess_1.5.1     spdep_0.5-29
coda_0.14-2        deldir_0.0-13      nlme_3.1-98        MASS_7.3-11
Matrix_0.999375-48
 [9] boot_1.2-43        spgwr_0.6-10       maptools_0.8-4
lattice_0.19-17    foreign_0.8-42     sp_0.9-79         

loaded via a namespace (and not attached):
[1] grid_2.12.1  tools_2.12.1

Here are the error messages when commands run for full sample:

            

      
      
      
    

        
Error: cannot allocate vector of size 922.5 Mb

            

      
      
      
    

        
Error: cannot allocate vector of size 922.5 Mb

            

      
      
      
    

        
Error: cannot allocate vector of size 922.5 Mb

            

      
      
      
    

        
Error: cannot allocate vector of size 922.5 Mb

            

      
      
      
    

        
If I run the same set of commands on a subset of the data, then the
command output is:

            

      
      
      
    

        
Error in solve.default(asyvar, tol = tol.solve) : 
  system is computationally singular: reciprocal condition number =
8.6909e-27

            

      
      
      
    

        
Error in solve.default(asyvar, tol = tol.solve) : 
  system is computationally singular: reciprocal condition number =
8.34845e-27

            

      
      
      
    

        
Error in solve.default(inf, tol = tol.solve) : 
  system is computationally singular: reciprocal condition number =
7.82721e-27

            

      
      
      
    

        
Error in solve.default(inf, tol = tol.solve) : 
  system is computationally singular: reciprocal condition number =
7.81985e-27

            

      
      
      
    

        
Suggestions?



-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no] 
Sent: Friday, March 25, 2011 4:12 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] Spatial Lag and Error Model

        
On Thu, 24 Mar 2011, Janmaat, John wrote:

        

            

      
      
      
    

        
You did not provide sessionInfo() output. If you update spdep, you'll
find 
that this issue (of poorly conditioned crossproduct matrices in the
stsls 
step) was addressed in December 2010, and the current release should
work.

The code and error messages from errorsarlm() and lagsarlm() will
probably 
also show other misunderstandings, but that can be addressed if you post

them.

Hope this helps,

Roger

            

      
      
      
    

        
zero

            

      
      
      
    

        
neighbour

            

      
      
      
    

        
a

            

      
      
      
    

        
This

            

      
      
      
    

        
zero

            

      
      
      
    

        
model

            

      
      
      
    

        
dropped.

            

      
      
      
    

        
I

            

      
      
      
    

        
data

            

      
      
      
    

        
each

            

      
      
      
    

        
the

            

      
      
      
    

        
use,

            

      
      
      
    

        
However,

            

      
      
      
    

        
is

            

      
      
      
    

        
spatial

            

      
      
      
    

        
precise

            

      
      
      
    

        
coefficients

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Fri, 25 Mar 2011, Janmaat, John wrote:

        

            

      
      
      
    

        
See ?errorrsarlm, method= argument. You should not expect to be able to 
use dense matrix methods with as many observations. For k nearest 
neighbours, I suggest LU for an exact result, perhaps MC for an 
approximation.

            

      
      
      
    

        
Ditto.

            

      
      
      
    

        
Ditto - ?lagsarlm

            

      
      
      
    

        
Ditto.

You may also see singularities because your RHS variables appear to be 
close to colinear.

            

      
      
      
    

        
Both of these come when inverting the asymmetric covariance matrix of the 
variables, read about why this happens on the function help pages - it is 
described in details, and may also concern the scaling of your variables.

            

      
      
      
    

        
Ditto.

            

      
      
      
    

        
Read the function help pages fully before you use the functions?

Roger

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Janmaat, John
    

    

      
      #
    

  


  

      
I have read the documentation, and have tried both the LU and MC
methods.  I continue to get a similar error.  With a subset of the data
and a simplified model, then it will execute the command.  In my case, I
am using both linear and quadratic terms for some independent variables
to capture curvature in the marginal effects.  These quadratic terms do
enter strongly significantly in the linear model, but seem to cause
problems for errorsarlm() and lagsarlm(), particularly when the full
sample is used.  

Thanks again for the comments.  

John.

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand at nhh.no] 
Sent: Saturday, March 26, 2011 4:03 AM
To: Janmaat, John
Cc: r-sig-geo at r-project.org
Subject: RE: [R-sig-Geo] Spatial Lag and Error Model

        
On Fri, 25 Mar 2011, Janmaat, John wrote:

        

            

      
      
      
    

        
one

            

      
      
      
    

        
See ?errorrsarlm, method= argument. You should not expect to be able to 
use dense matrix methods with as many observations. For k nearest 
neighbours, I suggest LU for an exact result, perhaps MC for an 
approximation.

            

      
      
      
    

        
Ditto.

            

      
      
      
    

        
Ditto - ?lagsarlm

            

      
      
      
    

        
Ditto.

You may also see singularities because your RHS variables appear to be 
close to colinear.

            

      
      
      
    

        
Both of these come when inverting the asymmetric covariance matrix of
the 
variables, read about why this happens on the function help pages - it
is 
described in details, and may also concern the scaling of your
variables.

            

      
      
      
    

        
Ditto.

            

      
      
      
    

        
Read the function help pages fully before you use the functions?

Roger

            

      
      
      
    

        
post

            

      
      
      
    

        
when

            

      
      
      
    

        
be

            

      
      
      
    

        
the

            

      
      
      
    

        
am

            

      
      
      
    

        
function

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Roger Bivand
    

    

      
      #
    

  


  

      
On Sat, 26 Mar 2011, Janmaat, John wrote:

        

            

      
      
      
    

        
You are still not providing verbatim commands and error messages; adding 
the output of traceback() after the error would also show more. Have you 
read LeSage & Pace (2009) on mixed numerical Hessians, and seen that the 
ML fitting functions have an argument called trs= which may be used to 
provide traces of the power series of W, generated by trW(), see ?trW to 
see how to do this, and choose the "MC" type. This may relieve the 
problem, but without seeing the complete command you are executing, it is 
very difficult to tell.

If the quadratic terms are of very large or very small numbers, you may 
prefer to rescale the linear to something more modest. You need to realise 
that computers carry out numerical operations on the data you have given - 
models that may look OK on paper will fall over when estimated for all 
kinds of numerical reasons. Did you consider using orthogonal polynomials 
from poly() instead of powers - this may be numerically more stable?

Roger