mikekestemont / echoes Goto Github PK

Low-priority enhancement: Add basic metadata to index (e.g. author, title, ...)

Title says it all.

Add "hybrid" language model from deepflow for Asibot-style sentence completion

For a plugin system, we might want to have a look at ZMQ, which offers a way to setup
microservices. A nice read is the following:

https://blog.apcelent.com/how-to-setup-microservices-python-zeromq-docker-example.html

We could start with playing around with a simple word synonym service, and try to plug
that into the API?

• the concordancer often yields empty snippets; we should filter these out.
• I added the prepare_elmo.py script which shows how we could we could add sentence-level representations. @fbkarsdorp any chance we can add these vectors to the whoosh index? We could also rely on faiss to get the sentence id, but retrieve it from the whoosh "sentence shop" (instead of the naive sqlite)?

• return synonyms as a word cloud
• return word-level synonyms (word2vec, fasttest)
• return phrase-level synonyms (elmo, bert)
• show a concordance (using woosh?)
• spot word repetitions and suggest alternatives as track changes