graphbrain / graphbrain

Language, Knowledge, Cognition

Graphbrain is an Artificial Intelligence open-source software library and scientific research tool. Its aim is to facilitate automated meaning extraction and text understanding, as well as the exploration and inference of knowledge. It is part of socsemics, a research project funded by the European Research Council and focused on the fragmentation in online public spaces.

The ambition of this project is not simply to contribute to advancing the state-of-the-art in AI, but principally to be an effective interdisciplinary research tool. More specifically, we are interested in advancing the possibilities of automated meaning extraction for research in fields such as Computational Social Sciences, Cognitive Science and the Humanities.

Graphbrain is built around a unifying concept: the Semantic Hypergraph (SH), which makes it possible to represent a natural language sentence such as “Einstein first published the theory of relativity in 1905” as an ordered, recursive hyperlink of the form:

The aim of SH and Graphbrain is to enable hybrid approaches, combining symbolic AI and statistical/connectionist Machine Learning methods. This is for two main reasons: to facilitate understandability and reproducibility for human researchers and to combine the strengths of both approaches.

Graphbrain is written in Python, to both take advantage and facilitate integration with the rich environment of scientific libraries available in this language. It is released under the highly permissive MIT open source license.

Installation instructions, the manual and more information can be found here: http://graphbrain.net.

graphbrain's Issues

types, subtypes...

Hi Telmo,

is possible to build a list of the types, subtypes and all related codes used in the program?

For example, I try to find in the code and the manual, but I still don't find what mean the "o" and the "x" in for example:

"tracking/Pd.sox.|pg—‐"

(manual, page 12)

Thank you in advance! :)
Miguel

Querying by hyperedge types

This is a feature request, or a request for clarification of the querying syntax. From what I can tell, there's no way to query edges based on their types and subtypes other than adding my own hyperedges relating atoms to these types. It's unclear to me what the point of even having subtypes is when they don't affect the structure and can't be directly queried.

For context, the specific issue I'm running into is number normalization. I expected to be able to use a pattern like (+/b.am/. @/c# @/c#) to match phrases like "seven hundred" and resolve them to a new atom 700/c#. After scouring the documentation and code, it seems like the only way to do this at the moment is to manually add hyperedges like (number/pd 7/c#) so they can be queried with (number/pd @), eg (+/b.am/. (number/pd @lhs) (number/pd @rhs))? But then that would require wrapping every mention of any number with this predicate. Am I missing something?

Issue in setup.py

line 87 is : with open('README.md', 'r')

it should be with open('README.md', 'r', encoding="utf8")

Otherwise it does not work on windows in French (my case) and other languages I guess

build on macOS: homebrew leveldb does not have rtti

I tried to install from source on MacOS (latest Big Sur, Intel) and had an issue when loading the plyvel .so, complaining about not finding the typeinfo in leveldb. It turns out the brew leveldb does not have typeinfo. Rebuilding plyvel with CFLAGS=-no-rtti solved the problem. Leaving this as reference for anybody running into the same issue, not sure if there's an action to take at the level of graphbrain.

Missing `graphbrain.meaning` module

Error

ModuleNotFoundError: No module named 'graphbrain.meaning'

I guess this is related to the missing of a __init__.py file inside the graphbrain/meaning folder, which doesn't become a package and it's not seen by the find_packages function in setup.py.

How to reproduce the behaviour

from graphbrain.agents import *

Your Environment

• Operating System: Linux-4.9.0-9-amd64-x86_64-with-debian-10.1
• Python Version Used: 3.8.1
• Graphbrain Version: 0.3.0

Does it support Chinese

Can't find model 'en_core_web_lg'. for spacy 3.* in colab

It seems spacy can work without problems, but when I try to run graphbrain, then on parser = create_parser(lang='en', lemmas=True) i got an error
`

OSError Traceback (most recent call last)

in ()
4
5
----> 6 parser = create_parser(lang='en', lemmas=True)
7 parse_results = parser.parse(text)
8

2 frames

graphbrain/parsers/parser_en.pyx in graphbrain.parsers.parser_en.ParserEN.init()

graphbrain/parsers/alpha_beta.pyx in graphbrain.parsers.alpha_beta.AlphaBeta.init()

/usr/local/lib/python3.7/dist-packages/spacy/util.py in load_model(name, **overrides)
173 elif hasattr(name, "exists"): # Path or Path-like to model data
174 return load_model_from_path(name, **overrides)
--> 175 raise IOError(Errors.E050.format(name=name))
176
177

OSError: [E050] Can't find model 'en_core_web_lg'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.`

I tried to link en_core_web_lg as en !python -m spacy link en_core_web_lg en --force, but same results.

pytest graphbrain fails: ImportError while importing test module

Hello,
I'm trying to install graphbrain. Installation seems to go well, but pytest fails, as shown below. My system is Linux Mint 20 Ulyana, I'm using python3.8. I'm following graphbrain installation instructions.
Might be some silly mistake on my part. Thanks in advance!

pytest graphbrain
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/martin/Documentos/programming/ontology/graphbrain/graphbrain
collected 0 items / 7 errors                                                                                                                                                                                                                 

=================================================================================================================== ERRORS ===================================================================================================================
_____________________________________________________________________________________ ERROR collecting graphbrain/neuralcoref/tests/test_neuralcoref.py ______________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/neuralcoref/tests/test_neuralcoref.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
______________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_corefs.py ______________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_corefs.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
____________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_hyperedge.py _____________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_hyperedge.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
____________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_hypergraph.py ____________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_hypergraph.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
______________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_logic.py _______________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_logic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
_____________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_patterns.py _____________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_patterns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
___________________________________________________________________________________________ ERROR collecting graphbrain/tests/test_permutations.py ___________________________________________________________________________________________
ImportError while importing test module '/home/martin/Documentos/programming/ontology/graphbrain/graphbrain/graphbrain/tests/test_permutations.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
graphbrain/__init__.py:1: in <module>
    from graphbrain.hyperedge import *
E   ModuleNotFoundError: No module named 'graphbrain.hyperedge'
========================================================================================================== short test summary info ===========================================================================================================
ERROR graphbrain/neuralcoref/tests/test_neuralcoref.py
ERROR graphbrain/tests/test_corefs.py
ERROR graphbrain/tests/test_hyperedge.py
ERROR graphbrain/tests/test_hypergraph.py
ERROR graphbrain/tests/test_logic.py
ERROR graphbrain/tests/test_patterns.py
ERROR graphbrain/tests/test_permutations.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 7 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================= 7 errors in 0.19s ==============================================================================================================

Create package and prepare for distribution

Shortest paths

Create a method for hypergraphs that, given two edges, e1 and e2, finds the set of shortest paths from one to the other.

To illustrate, this edge is itself the shortest path between e1 and e2:
(is/P e1 e2)

Or, if only the following two edges exist, they are the shortest path between e1 and e2:

(is/P e1 e3)
(likes/P e3 e2)

Or, there could be multiple shortest paths:

(is/P e1 e3)
(likes/P e3 e4)
(hates/P e3 e4)
(equals/P e4 e2)

And so on.

Notice that naive solutions to this problem lead to combinatorial explosion. Depending also on the topology of the hypergraph in question, the search space could increase exponentially with the length of the shortest path.

Proposals

Several approaches are possible here, to be decided:

• Implement the naive method for now, with a simple heuristic (e.g. maximum number of visited nodes) to give up;
• Implement some non-naive approach that uses optimizations and heuristics to tame the combinatorial explosion, possibly not guaranteeing complete results for large shortest path lengths;
• Implement a meta-solution that can be extended with new approaches in the future. Above approaches (naive, mixed) can be included. This makes sense if we figure that the problem has sufficient depth (e.g. there could be different notions of neighborhood, such as a recursive one).

Related issues

#21

Update terminology

According to the upcoming paper.

graphbrain agent ... "run" is still implemented?

Hi there!
Dear Telmo and colleagues, I think you did a terrific job with Graphbrain. A lot of potentialities :)
I have one question.
Tthe "run" command for graphbrain agents, is still working?
I try to implement using the examples of the manual, but always give me error.

For example, in an venv, writing:
"graphbrain --hg ai.db --url https://en.wikipedia.org/wiki/Artificial_intelligence --agent wikipedia run"
I obtain:
"command: run

agent: wikipedia
hypergraph: ai.db
language: en
url: https://en.wikipedia.org/wiki/Artificial_intelligence

Traceback (most recent call last):
File "/home/mfg/graphbrain/qom2/bin/graphbrain", line 8, in
sys.exit(cli())
File "/home/mfg/graphbrain/qom2/lib/python3.8/site-packages/graphbrain/cli.py", line 171, in cli
raise RuntimeError('Unknown command: {}'.format(args.command))
RuntimeError: Unknown command: run"

Thank you in advance for any help!
Miguel

Remove dependency from pandas

How to attack the hypergraph

Hi there. I am interested in doing the following information extraction process:

If A is connected to B and C is connected to B, how can I attack the graph to know that the shorter path from A to C is through B? In other words, how do I find paths of interest? I coud not find any function to do it.

Thanks a lot. This tool seems to be great. Congratulations for your work.
Ivó

Graphbrain installation fails when using Python 3.8.3 and Cython version 0.29.21

I am trying to install graphbrain with latest Python but I am getting multiple errors.
Here is part of the error log messages:
graphbrain/hyperedge.c:330:82: note: expanded from macro '__Pyx_PyCode_New'
PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
~~~~~~~~~~ ^~~~
/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8/code.h:122:12: note: 'PyCode_New' declared here
PyAPI_FUNC(PyCodeObject ) PyCode_New(
^
graphbrain/hyperedge.c:17470:261: error: too many arguments to function call, expected 15, have 16
__pyx_codeobj__63 = (PyObject)__Pyx_PyCode_New(1, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__62, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_graphbrain_hyperedge_pyx, __pyx_n_s_roots, 315, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__63)) __PYX_ERR(0, 315, __pyx_L1_error)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
graphbrain/hyperedge.c:330:82: note: expanded from macro '__Pyx_PyCode_New'
PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
~~~~~~~~~~ ^~~~
/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8/code.h:122:12: note: 'PyCode_New' declared here
PyAPI_FUNC(PyCodeObject *) PyCode_New(
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'gcc' failed with exit status 1

I think it is related to the changes in Cython, like here:
cython/cython#2938

Thanks

Installation fails (M1 Mac problem)

edit: solution below

pip install graphbrain fails inside new virtual environment.

Platform: Apple Silicone (M1)
OS: macOS Montery (12.5)
Python: 3.10.5 (installed via pyenv)

...
            fatal error: too many errors emitted, stopping now [-ferror-limit=]
            1 warning and 20 errors generated.
            error: command '/usr/bin/clang' failed with exit code 1
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for spacy
      Failed to build spacy
      ERROR: Could not build wheels for spacy, which is required to install pyproject.toml-based projects
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Installation of spacy 2.x versions also fails, as do numpy 1.1x versions.

---

Installing from source (pip install -e .) and removing the restriction of spacy<=3.0.0 (pobably breaking, but just to test it) avoids these errors but, introduces an error when installing plyvel.
This error could be fixed by doing brew install leveldb and CFLAGS="-I/opt/homebrew/include -L/opt/homebrew/lib" pip install -e ., but then an error at import time occurs:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/Users/max18768/Documents/Privat/graphbrain_playground/playground.ipynb Cell 1 in <cell line: 1>()
----> [1](vscode-notebook-cell:/Users/max18768/Documents/Privat/graphbrain_playground/playground.ipynb#W0sZmlsZQ%3D%3D?line=0) import graphbrain.parsers as parsers

File ~/Documents/Privat/graphbrain_playground/graphbrain/graphbrain/parsers/__init__.py:4, in <module>
      1 from importlib import import_module
      3 from .nlp import print_tree
----> 4 from .parser_en import ParserEN
      5 from .text import edge_text
      8 def create_parser(lang=None, parser_class=None, lemmas=False, corefs=False,
      9                   beta='repair', normalize=True, post_process=True):

File graphbrain/parsers/parser_en.pyx:3, in init graphbrain.parsers.parser_en()

ImportError: cannot import name hedge

Upload to the Python Package Index

Remove dependency from igraph

Atoms enrichment

Hello,

First of thanks for the great package, it has been very useful. I was wondering is there a way to get NLP information from hyper graph edges. In word2vec similarity example there is

graphbrain/graphbrain/stats/word2vec_similarity.py

Line 2 in 6294705

from graphbrain.nlp.enrich_edge import enrich_edge

but there is no package named nlp in graphbrain? Is there a way to achieve this?

create manylinux wheel

https://github.com/pypa/manylinux

when i run file ./examples/parse-text.py,i met problem below

print_tree(parse['spacy_sentence'].root)
TypeError: string indices must be integers

Parser issue with compound nouns and conjunctions

I've noticed an issue with the parser which I've reduced to a simple test case for the sentence:

South Korea, Singapore, Hong Kong and Taiwan.

I would expect the parser to generate an edge for (+b/. south korea) but it does not.

Test script to reproduce:

from graphbrain import *
from graphbrain.parsers import *

if __name__ == '__main__':
    text = "South Korea, Singapore, Hong Kong and Taiwan."

    parser = create_parser('en', lemmas=False, resolve_corefs=False)
    parse_res = parser.parse(text.strip())

    for parse in parse_res['parses']:
        print_tree(parse['spacy_sentence'].root)
        print()
        print(parse['main_edge'])

Outputs:

* korea/Korea/NNP (ROOT) {GPE}
 +-- < south/South/NNP (compound) {GPE}
 +-- > ,/,/, (punct) 
 +-- > singapore/Singapore/NNP (conj) {GPE}
 |   +-- > ,/,/, (punct) 
 |   +-- > kong/Kong/NNP (conj) {GPE}
 |       +-- < hong/Hong/NNP (compound) {GPE}
 |       +-- > and/and/CC (cc) 
 |       +-- > taiwan/Taiwan/NNP (conj) {GPE}
 +-- > ././. (punct) 

(+/b.ammm/. south/cp.s/en korea/cp.s/en singapore/cp.s/en (and/b+/en (+/b.am/. hong/cp.s/en kong/cp.s/en) taiwan/cp.s/en))

Note that the edge for Hong Kong is however correctly generated as (+/b.am/. hong/cp.s/en kong/cp.s/en)

Can't get started with CLI

Hi,

I have been trying to install graphbrain on my Mac and after getting around the leveldb issue I now get the following error message on my first attempt.

$ graphbrain --hg test1 create


                        _     _               _       
                       | |   | |             (_)      
   __ _ _ __ __ _ _ __ | |__ | |__  _ __ __ _ _ _ __  
  / _` | '__/ _` | '_ \| '_ \| '_ \| '__/ _` | | '_ \ 
 | (_| | | | (_| | |_) | | | | |_) | | | (_| | | | | |
  \__, |_|  \__,_| .__/|_| |_|_.__/|_|  \__,_|_|_| |_|
   __/ |         | |                                  
  |___/          |_|                                  



command: create

hypergraph: test1

creating hypergraph...
Traceback (most recent call last):
  File ".../bin/graphbrain", line 10, in <module>
    sys.exit(cli())
  File ".../lib/python3.7/site-packages/graphbrain/cli.py", line 66, in cli
    wrapper(_cli, command=True)
  File ".../lib/python3.7/site-packages/graphbrain/cli.py", line 50, in wrapper
    fun(args)
  File ".../lib/python3.7/site-packages/graphbrain/cli.py", line 59, in _cli
    cmd_module.run(args)
  File ".../lib/python3.7/site-packages/graphbrain/commands/create.py", line 6, in run
    hypergraph(args.hg)
NameError: name 'hypergraph' is not defined

Any ideas on how I can get past that? Looks like a project I have been looking for.

Cheers,
-max

Improve notebook hyperedge visualization

Host our own spacy models

So that we can be sure that nothing changes unexpectedly.

Installing codebase on an M1 mac

I was following your installation instructions on an M1 mac. I found that I needed to follow this to get it to work: wbolster/plyvel#100 (comment)

I think this stems from the libraries in use, not your code, but it may save some developers time if you link to this on your installation instructions.

Fix test warning

(venv) ➜ graphbrain git:(master) ✗ ./setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.

Remove dependency from nltk

python -m pip install graphbrain errors

python -m pip install graphbrain gives me:

ERROR: Command errored out with exit status 1: command: 'E:\WPy-3710\python-3.7.1.amd64\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-a1ivf6gk\\graphbrain\\setup.py'"'"'; __file__='"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-a1ivf6gk\\graphbrain\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: C:\Users\User\AppData\Local\Temp\pip-install-a1ivf6gk\graphbrain\ Complete output (5 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\User\AppData\Local\Temp\pip-install-a1ivf6gk\graphbrain\setup.py", line 24, in <module> with open('VERSION', 'r') as version_file: FileNotFoundError: [Errno 2] No such file or directory: 'VERSION' ---------------------------------------- ERROR:Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Running 64 bit WinPython on 64 bit Windows 10
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32