The documentation of entity-fishing is available here.
Presentation of entity-fishing at WikiDataCon 2017 for some design and implementation descriptions.
entity-fishing performs the following tasks:
-
entity recognition and disambiguation against Wikidata in a raw text or partially-annotated text segment,
-
entity recognition and disambiguation against Wikidata at document level, for example a PDF with layout positioning and structure-aware annotations,
-
search query disambiguation (the short text mode) - bellow disambiguation of the search query "concrete pump sensor" in the service test console,
-
weighted term vector disambiguation (a term being a phrase),
entity-fishing is a work-in-progress! Latest release version is 0.0.3
.
This version supports English, French, German, Italian and Spanish, with an in-house Named Entity Recognizer for English and French. For this version, the knowledge base includes 37 million entities from Wikidata.
Runtime: on local machine (Intel Haswel i7-4790K CPU 4.00GHz - 8 cores - 16GB - SSD)
-
800 pubmed abstracts (172 787 tokens) processed in 126s with 1 client (1371 tokens/s)
-
4800 pubmed abstracts (1 036 722 tokens) processed in 216s with 6 concurrent clients (4800 tokens/s)
-
136 PDF (3443 pages, 1 422 943 tokens) processed in 1284s with 1 client (2.6 pages/s, 1108.2 tokens/s)
-
816 PDF (20658 pages, 8 537 658 tokens) processed in 2094s with 6 concurrent clients (9.86 pages/s, 4077 tokens/s)
Accuracy: f-score for disambiguation only between 76.5 and 89.1 on standard datasets (ACE2004, AIDA-CONLL-testb, AQUAINT, MSNBC) - to be improved in the next versions.
The knowledge base contains more than 1 billion objects, not far from 15 millions word and entity embeddings, however entity-fishing will work with 3-4 GB RAM memory after a 15 second start-up for the server (but please use SSD!).
Distributed under Apache 2.0 license. The dependencies used in the project are either themselves also distributed under Apache 2.0 license or distributed under a compatible license.
Main author and contact: Patrice Lopez ([email protected])
entity-fishing is developed by SCIENCE-MINER with contributions of Inria Paris.
Inria contributors were supported by the H2020 HIRMEOS, IPERION-CH and DESIR EU projects.