camoverride / question_answering Goto Github PK

make this a package that you can install with pip

• confirm that BERT-SQUAD is fine-tuned on wikipedia!
• check out huggingface models for a smaller model -- might have to fine-tine this
• investigate wiki's native elasticsearch engine
• find cheap method for calling BERT's tokenizer instead of my whitespace split (TODO)
• chunk articles with overlap in case answer lies at the boundary (this is unlikely though) (TODO)
• find better decision criteria than simply sum of the max from start/end tensors. (TODO)

put the tokenizer on model server (doc) and then:

• modify Answerer._get_tokens so that it uses BERT tokenizer instead of whitespace tokenization
• chunk articles with overlap in case answer lies at the boundary (this is unlikely though)
• warm up tokenizer model and BERT model

Code should be run like this

from question_answering import Answerer

# Gives a lot of info
ans = answerer.answer_question("What is the population of Canada?")

# Parse it
ans["answer"]["answer"]

However, there should be several options for configuring this locally:

• 1. it's a python package that you pip install. Large model is downloaded and is spun up/town down at every request. This is simple but slow.
• 2. It's a python package that you pip install. BUT you then launch a model server locally that runs in the background.
• 3. It's a python package that you pip install. You run the model server in the background. Also, you download wikipedia and query that locally instead of going over the web - this is 80 gigs of data!

This is how a model server should be configured, after the docker model server is running:

# Import QA without loading the model into memory.
import question_answering as qa
qa.config("model_address" = "localhost:4000")

# Gives a lot of info
ans = qa.Answerer().answer_question("What is the population of Canada?")

# Parse it
ans["answer"]["answer"]