noseworm / convai Goto Github PK

Let's take squad dataset and build the following:

given article, generate one or more questions
This could be done as a preprocessing step just once in the whole conversation if we store all the generated responses for the length of the conversation

Extract facts from Wikipedia using Elasticsearch api

Implementation ideas:

• Download the latest wikipedia dump
• Run Elasticsearch server
• Query by entity
• Return first line of the top article

The wikipedia dump can be found here. Although, given the huge size of this dump, I don't know whether it would be feasible for us to wrap this within our docker container. Easier way to tackle this problem would be to use the python wikipedia api, but as per the convai rules we are not allowed to place external api calls.

• Verify with the organizers.

The docker file is now at 51GB, check if the correct version is getting pushed.

I think we should probably have anaphora resolution as a preprocessing step? Idk how hard this would be but I wanted to see what people thought about this?

Joelle suggested us to see if we can modify the beam sampling in HRED to account for the user feedback, possibly trickling down the user rating for the conversation to sample the next best sentence.

We already have Dual Encoder retrieval model, but the responses given by Rosemary in Ethics paper seems pretty good. can we quickly add the existing vhred model? (should be straightforward)

Might be a bit of hack, but since we have 2500 human-human dialogs (x20 average), we could train a dual encoder setup to rank human responses!

Could be useful to implement a topic classification model using fasttext to extract the topics which are being talked in the article about.

Implementation plan:

• Get a set of general topics from wibi taxonomy
• Get wikipedia articles for each of the above topics, including their children (prune to children having at least 10 child nodes)
• Train fasttext
• Evaluate

Let's try to integrate a sentiment classifier as a feature for the ranker network!
see: https://github.com/cjhutto/vaderSentiment
In the MILA bot paper they have a sentiment feature in their SupervisedAMT model I think

Let's add a little hack to the HRED model to avoid repeating itself in the same sentence

List of features to implement / implemented for the RankerNN. If you have worked / want to work on a particular feature claim it beside the point. When the bullet point is done mark it done with an x in between the brackets like this [x]. The list is extracted from this google doc.

• Averaged word embeddings (dimension : 300)
  • candidate response (Nicolas)
  • last user turn (Nicolas)
  • last k turns (Nicolas)
  • last q user turns (Nicolas)
  • article (Nicolas)
• Embedding similarity metrics w.r.t the candidate response
  • last user turn (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • last k turns (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • last k turns without stop words (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • last q user turns (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • last q user turns without stop words (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • article (Nicolas)
    • greedy score
    • average score
    • Extrema score
  • article without stop words (Nicolas)
    • greedy score
    • average score
    • Extrema score
• word/entity overlap:
  • non-stop word overlap (Nicolas)
  • bigram overlap (Koustuv)
  • trigram overlap (Koustuv)
  • entity overlap (Koustuv)
• generic turn (Nicolas)
• word presence:
  • “Wh”-word (Koustuv)
  • intensifier word (Koustuv)
  • confusion word (Koustuv)
  • Profanity list (Koustuv)
  • negation word (Nicolas)
• lengths:
  • dialogue length (Koustuv)
  • last user turn length
  • response length
  • article length (Koustuv)
• dialogue act
• Sentiment score (https://github.com/cjhutto/vaderSentiment - https://pypi.python.org/pypi/vaderSentiment) (Koustuv)

Incorporate ALICE BOT

Implementation plan:

• Get Alicebot AIML files. Consult Julian about the files they used
• Load Alicebot as a new model (simple python import should work)

List of AIML files we can use can be found in the MILABOT repo. Although would be great to find some open resource for this.

In order to implement this, first install python package aiml, then following this tutorial, load the aiml files. Save the file state in "brain" for faster processing.

While this is a good addition, I have some apprehensions in using this, as its mostly rule based = less innovation

We have a followup questions model but it only generates very basic one-liner questions ("what","why","huh"). Since we have documents and articles I propose we use something like this : Neural QA (accompanying paper) . The original code, again, is in Torch7. Would it be useful to use it as is or port it quickly to PyTorch?

In order to maximize immediate response, we could create an evaluation dataset, where given a context dialogs and article, we provide all the candidate responses, and ask user to rate / rank them.

detect when user asks something related to the article.
example of training set: squad with (article, question, flag) triples where question is either related (flag=1) or not (flag=0) to the article