update the ttprime computation with tqdm + desactivating flag, etc...

ttprime should be computed after post-processing; also the optimal-scale selection can only be applied when ttprime was computed

How to reproduce:

1. Create and activate a new conda (v4.8.3) environment with Python (v3.8.3) installation.
2. Run $ conda install pybind11 pyparsing six
3. Clone the PyGenStability repository
4. Run $ pip install . while in the cloned repository
5. Copy the contents of the subfolder examples to the top level directory
6. Run $ python simple_example.py

Traceback (most recent call last):
  File "simple_example.py", line 25, in <module>
    simple_test()
  File "simple_example.py", line 14, in simple_test
    all_results = run(graph)
  File "/home/benedikt/pygenstability/PyGenStability/pygenstability/pygenstability.py", line 119, in run
    louvain_results = run_several_louvains(
  File "/home/benedikt/pygenstability/PyGenStability/pygenstability/pygenstability.py", line 251, in run_several_louvains
    list(pool.imap_unordered(worker, range(n_runs), chunksize=chunksize))
  File "/home/benedikt/anaconda3/envs/pygen_env/lib/python3.8/multiprocessing/pool.py", line 448, in <genexpr>
    return (item for chunk in result for item in chunk)
  File "/home/benedikt/anaconda3/envs/pygen_env/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
AttributeError: module 'pygenstability.generalized_louvain' has no attribute 'run_louvain'

use itertools and compute only lower triangular part

The error that is raised on a Mac with M2 Pro chip is TypeError: data type 'float128' not understood. Can we somehow fix this?

the code is not tested, some tests would be nice

A mess at the moment

Precompute the Laplacian and null model once at the start, not every time call of exponential

On some windows, it seems pthreads.h is missing, we'll investigate more on that.

On windows, it may be hard to install do to pthread.h issue, so maybe we can allow to not instal louvain, and rather use leiden.

The current scale selection criterion is not reliable at the margins of the domain of Markov scale t due to:

1. Cut-off through windowing
2. The diagonal count of low values in the NVI(t,t') necessarily leads to smaller values at the margins because the diagonal is smaller. Hence, 1-NVI(t,.) tends to produce anomalous local minima at the margins.

While 1) is perhaps acceptable, Mauricio and me discussed that we should come up with a solution for 2). I will think about this and try a couple of things.

There are other big libraries that could include RMST-RBS:

• Community Detection https://github.com/GiulioRossetti/cdlib
  • https://github.com/igraph/igraph and https://github.com/igraph/python-igraph
  • https://github.com/networkx/networkx
• Node/Graph Embedding https://github.com/benedekrozemberczki/karateclub
• Node/Graph Embedding https://github.com/shenweichen/GraphEmbedding

in the process of install pygenstability, i have the following problems：
can you help me to solve it?
building 'pygenstability.generalized_louvain' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iextra -ID:\Install_software\Anaconda\envs\tensorflow\lib\site-packages\pybind11\include -ID:\Install_software\Anaconda\envs\tensorflow\include -ID:\Install_software\Anaconda\envs\tensorflow\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /EHsc /Tppygenstability/generalized_louvain/generalized_louvain.cpp /Fobuild\temp.win-amd64-3.7\Release\pygenstability/generalized_louvain/generalized_louvain.obj /EHsc /bigobj -std=c++11
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
generalized_louvain.cpp
extra\lemon/bits/lock.h(24): fatal error C1083: 无法打开包括文件: “pthread.h”: No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit status 2

replacve the equal signe here https://github.com/ImperialCollegeLondon/PyGenStability/blob/master/pygenstability/pygenstability.py#L33
with a drop 0 form sparse matrix for possible speedup

Hello,
I'am trying to use the package on a big network (around 170000 nodes).
However, when I call the contractor:

defining the constructor externally

directed_constructor = constructors.load_constructor('directed', adjacency, alpha=0.85)

I get the following error:
Unable to allocate 241. GiB for an array with shape (179783, 179783) and data type float6

Is there a way to avoid it?

Moreover, could i give as input a weighted adj matrix?

Thanks a lot

We are currently forcing pyyaml as a dependency, however, some people are experiencing a bug that PyYAML cannot be uninstalled (to then update) because its a distutils installed project.

Seems ok to use, and provides interactive plots so that we can click on the community structure for example.

Buggy at the moment

Due to cpp, it is not trivial to deploy on pypi, we need to work on that

try to see on how large graphs we can run the code

so they can be use standalone more easily

Make a nice documentation with an associated website (using sphinx)

As pointed out in the code, we might be able to come up with better ways to compute matrix exponentials repeatedly:

just ordering entries by cluster ids

In the line 208 of plotting.py:matplotlib.use("TkAgg")
Not everyone uses the tkagg.

Maybe before matplotlib.use("Agg"),
let b = matplotlib.get_backend()
then matplotlib.use(b), instead of matplotlib.use("TkAgg")

Make sure cli is up to date, and does not use pickle, but better format for large graphs

The value for modularity/stability is behaving strangely, need to investigate that.

make it pip installable once it is ready

the sankey diagram ploting function works but is rudimentary. Some work to improve it is needed