ziqi-li / fastgwr

Fast Geographically Weighted Regression (FastGWR)

FastGWR

A command line tool for fast parallel computation of Geographically Weighted Regression models (GWR and MGWR).

New feature:

Multi-scale GWR model added!

Installation:

The fastgwr program is dependent on mpi4py package and a working MPI implementation. The easiest way to install both dependencies is to use conda:

$ conda install mpi4py

By installing mpi4py, conda will also install an MPI implementation based on your computer system (OpenMPI for Mac/Linux; MPICH/MS-MPI for Windows). Users may want to check whether the MPI implementation is successfully installed and is on your path by running the mpiexec command. Then the fastgwr program can be installed from PyPi:

$ pip install fastgwr

After sucessful installation, users can test the functionalities from the command line by running:

# Using zillow sample data for testing MGWR model fitting.
$ fastgwr testgwr

or

# Using zillow sample data for testing MGWR model fitting.
$ fastgwr testmgwr

Examples

Example call to the fastgwr to fit GWR model:

$ fastgwr run -np 4 -data input.csv

Example call to the fastgwr to fit MGWR model:

$ fastgwr run -np 4 -data input.csv -mgwr

where:

-np 4             Number of processors (e.g. 4).
-data input.csv   Input data matrix. (e.g. input.csv)
                  Can also be an URL (e.g. https://raw.github.com/
                  Ziqi-Li/FastGWR/master/Zillow-test-dataset/zillow_1k.csv)
-out results.csv  Output GWR results matrix including local parameter 
                  estimates, standard errors and local diagnostics.
-adaptive/-fixed  Adaptive Bisquare kernel (defualt) or Fixed Gaussian kernel.
-bw 1000          Pre-defined bandwidth parameter. If missing, it will
                  search (golden-section) for the optimal bandwidth and use
                  that to fit the GWR model.
-minbw 45         Lower bound in golden-section search. (e.g. 45)
-mgwr             Fitting an MGWR model.
-chunks           Number of chunks for MGWR computation (set to a larger 
                  number to reduce memory footprint).
-estonly          Allowing MGWR to output parameter estimation only.

The input needs to be prepared in this order:

	X-coord	y-coord	y	X1	X2	X3	Xk
	...	...	...	...	...	...	...
	...	...	...	...	...	...	...

where:
X-coord: X coordinate of the location point
Y-coord: Y coordinate of the location point
y: dependent variable
X1...Xk: independent variables

See the example Zillow datasets in the repository.

Results Validation

The results are validated against the mgwr, which can be seen in the notebooks here.

Citations

This program is developed based on these two papers:

FastGWR

Li, Z., Fotheringham, A. S., Li, W., Oshan, T. (2019). Fast Geographically Weighted Regression (FastGWR): A Scalable Algorithm to Investigate Spatial Process Heterogeneity in Millions of Observations. International Journal of Geographic Information Science. doi: 10.1080/13658816.2018.1521523.

FastMGWR

Li, Z., & Fotheringham, A. S. (2020). Computational improvements to multi-scale geographically weighted regression. International Journal of Geographical Information Science, 34(7), 1378-1397.

fastgwr's Issues

[Solved][Ubuntu/Mint] FastGWR Not Working

I had issues with FastGWR on two Linux Mint (Ubuntu derivative) machines. One was Linux Mint 19.3 (tricia/bionic) with python 3.8.0 and the other was Linux Mint 20 (ulyana/focal) with python 3.8.5. fastgwr --version worked, but when testing/running the software, I was getting various errors about mpiexec despite mpiexec being installed. The error ended up being that I needed to update alternatives for python because both systems had python2 and python3 installed.

To check if you have the same problem run:

>>> update-alternatives --list python
update-alternatives: error: no alternatives for python

If you get the error above, follow the steps in this article to update alternatives to set Python to the correct version. I was using Python 3.8 so for both machines the solution was:

>>> sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 1
update-alternatives: using /usr/bin/python3.8 to provide /usr/bin/python (python) in auto mode

This comand sets python3.8 to your first priority alternative for python. To check this worked, run

>>> update-alternatives --list python
/usr/bin/python3.8

You should get the correct python3 as the result of the command. To check that this solved the issue with FastGWR run:

>>> fastgwr testgwr
Testing GWR with zillow data:
------------------------------------------------------------
Starting FastGWR with 2 Processors
Spatial Kernel: Adaptive Bisquare
Data Input Path: https://raw.github.com/Ziqi-Li/FastGWR/master/Zillow-test-dataset/zillow_1k.csv
Output Result Path: fastgwr_rslt.csv
Intercept: True
Optimal Bandwidth Searching...
Range: 50 1000
BW, AICc 413.0 12499.779288982281
BW, AICc 637.0 12531.633782051933
BW, AICc 274.0 12474.956021862543
BW, AICc 189.0 12431.308996341173
BW, AICc 136.0 12409.551843948328
BW, AICc 103.0 12408.264954026834
BW, AICc 83.0 12421.22851668721
BW, AICc 116.0 12405.67119279391
BW, AICc 123.0 12406.489388497328
BW, AICc 111.0 12405.452378901939
BW, AICc 108.0 12405.964286829414
BW, AICc 113.0 12405.540369991997
BW, AICc 110.0 12405.334206357205
BW, AICc 109.0 12405.904039101335
Fitting GWR Using Bandwidth: 110.0
Diagnostic Information:
AICc: 12405.334206357205
ENP: 112.68000986186829
R2: 0.7543256014825577
Total Time Elapsed: 3.58 seconds
------------------------------------------------------------

Error running example code

Disregard

Fast GWR error

Hello,

Thank you for sharing the great code of Fast GWR.

I have used my own file like below.

zillow = pd.read_csv("model.csv",sep=',')
zillow.head()

But I have got the below error. Please let me know how to fix it.

#Running FastGWR 1 Processor with auto bandwidth search
!fastgwr run -np 1 -data model.csv -out results_adap.csv -adaptive -constant

Usage: fastgwr run [OPTIONS]
Try 'fastgwr run --help' for help.

Error: no such option: -c

Is it possible to consider adding some method involve time scale?

李博士 你好：
为了保证我的表述准确，请允许我先以中文叙述，后我会以英文再次叙述我的问题以保证讨论能被尽可能多的人理解。
考虑到大量的数据往往是存在时间动态的，但mgwr或gwr并没有解决这一点，而当前已有的广泛使用的模型gtwr考虑了时间但没有考虑不同自变量的多尺度问题，也因此有了mgtwr方法，该方法在github上同样有共享库（https://github.com/sunkun1997/mgtwr）。
然而正如fastgwr方法所讨论的核心问题一样，我现在正试图处理百万级别时空Postion的数据，该方法的应用包括gtwr方法在内都要求几十t的内存，这无疑是不可能的。
请问是否可以在您的工具里添加gtwr方法或更进一步添加mgtwr方法？
谢谢你的阅读，期待您的回复。
Hello, Dr. Li.
In order to make sure that my statement is accurate, please allow me to state my question in Chinese first, and then I will state my question again in English to ensure that the issue can be understood by as many people as possible.
Considering that a large amount of data is often time-dynamic, mgwr or gwr does not solve this problem, and the existing widely used model gtwr considers time but does not consider the multi-scale problem of different independent variables, hence the mgtwr method. The method on making the same Shared library in github(https://github.com/sunkun1997/mgtwr).
However, as the core problem of the fastgwr method is discussed, I am now trying to deal with the data of the million-level space-time Postion, and the application of this method, including the gtwr method, requires tens of terabytes of memory, which is undoubtedly impossible.
Is it possible to add gtwr methods to your tool or go further and add mgtwr methods?
thanks.
XIaoran Yin

Add Spherical Coordinate option

Is it possible to add an argument for spherical coordinates like mgwr has? I see that mgwr is computing the haversine and that FastGWR is using cdist, which I believe can be passed "metric=haversine".

FastGWR fails with singular matrices (is this intentional?)

I noticed that the code fails if there is a singular matrix because you're using inverse rather than pseudo-inverse.

I wanted to check if this was intentional as I'm not an expert on GWR. If not, I'd be happy to make a pull request that catches the singular matrix error and then uses the pseudo-inverse.

Thanks!

GPU

Does this version have a version that calls the server GPU