Dear all, Is there an option to shorten the running time for the gwr() function, similar to the ?LU? method for lagsarlm() in the spdep package? Because I have a model with roughly 500?000 observations, and the running time at the moment is quite long, respectively it has not yet terminated. Thank you for your help in advance. Best regards, Raphael Mesaric
Question spgwr package - running time gwr()
4 messages · Roger Bivand, Raphael Mesaric
On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:
Dear all, Is there an option to shorten the running time for the gwr() function, similar to the ?LU? method for lagsarlm() in the spdep package? Because I have a model with roughly 500?000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.
What are you actually doing? Why did you choose GWR? Are you fitting a GWR with 500K observations, or have you fitted a GWR with many fewer observations, and are now rendering that fitted model with 500K fit points? GWR is only for detecting possible non-stationarity or similar mis-specification in moderately sized data sets. Trying to fit with 500K gives a dense hat matrix of 500K x 500K, which is imposssible (or were it possible would be uninformative). Think of 5K as a sensible maximum if GWR is condidered sensible at all. I would think that finding a bandwidth is impossible too. Roger
Thank you for your help in advance. Best regards, Raphael Mesaric
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no https://orcid.org/0000-0003-2392-6140 https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
Dear Roger, Thank you very much for your reply. My thesis supervisor told me to use GWR to explore small-scale differences and test the consistency of the SAR models. So, I tried to fit the GWR model with all my observations. However, if I understood you correctly, I would have to choose smaller sections of the grid and try to fit a GWR there. What I am not sure about how to do this in order to get informative results. As for the SAR model, the dependent variable consists n blocks where the m-th entry of each block corresponds to the m-th cell of my grid. So, if I have to reduce the grid to about 5K entries, I would need to take a subset of roughly 250 cells and then take all the 20 observations for each of these cells? And then try to fit a GWR there? Best, Raphael
Am 17.05.2019 um 14:20 schrieb Roger Bivand <Roger.Bivand at nhh.no>: On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:
Dear all, Is there an option to shorten the running time for the gwr() function, similar to the ?LU? method for lagsarlm() in the spdep package? Because I have a model with roughly 500?000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.
What are you actually doing? Why did you choose GWR? Are you fitting a GWR with 500K observations, or have you fitted a GWR with many fewer observations, and are now rendering that fitted model with 500K fit points? GWR is only for detecting possible non-stationarity or similar mis-specification in moderately sized data sets. Trying to fit with 500K gives a dense hat matrix of 500K x 500K, which is imposssible (or were it possible would be uninformative). Think of 5K as a sensible maximum if GWR is condidered sensible at all. I would think that finding a bandwidth is impossible too. Roger
Thank you for your help in advance. Best regards, Raphael Mesaric
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Roger Bivand Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no https://orcid.org/0000-0003-2392-6140 https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
1 day later
On Fri, 17 May 2019, Raphael Mesaric wrote:
Dear Roger, Thank you very much for your reply. My thesis supervisor told me to use GWR to explore small-scale differences and test the consistency of the SAR models.
It happens. From your problem description, I'm unsure whether GWR or SAR are appropriate methods, and would suggest the mixed-model and multilevel literatures. Mapable IID and spatially structured random effects may be more appropriate. Roger
So, I tried to fit the GWR model with all my observations. However, if I understood you correctly, I would have to choose smaller sections of the grid and try to fit a GWR there. What I am not sure about how to do this in order to get informative results. As for the SAR model, the dependent variable consists n blocks where the m-th entry of each block corresponds to the m-th cell of my grid. So, if I have to reduce the grid to about 5K entries, I would need to take a subset of roughly 250 cells and then take all the 20 observations for each of these cells? And then try to fit a GWR there? Best, Raphael
Am 17.05.2019 um 14:20 schrieb Roger Bivand <Roger.Bivand at nhh.no>: On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:
Dear all, Is there an option to shorten the running time for the gwr() function, similar to the ?LU? method for lagsarlm() in the spdep package? Because I have a model with roughly 500?000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.
What are you actually doing? Why did you choose GWR? Are you fitting a GWR with 500K observations, or have you fitted a GWR with many fewer observations, and are now rendering that fitted model with 500K fit points? GWR is only for detecting possible non-stationarity or similar mis-specification in moderately sized data sets. Trying to fit with 500K gives a dense hat matrix of 500K x 500K, which is imposssible (or were it possible would be uninformative). Think of 5K as a sensible maximum if GWR is condidered sensible at all. I would think that finding a bandwidth is impossible too. Roger
Thank you for your help in advance. Best regards, Raphael Mesaric
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Roger Bivand Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no https://orcid.org/0000-0003-2392-6140 https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
Roger Bivand Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no https://orcid.org/0000-0003-2392-6140 https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en