dylan-fan / keras_qa

Keras solution to the bAbI tasks using recurrent neural networks - merged as an example into Keras mainline

Keras question answering for bAbi

Note: This code has been merged as an example into Keras -- see babi_rnn.py

This repository contains Keras code to train two recurrent neural networks based upon a story and a question. The resulting merged vector is then queried to answer a range of bAbI tasks.

An example from the first task, QA1, is below:

1 Mary moved to the bathroom.
2 John went to the hallway.
3 Where is Mary? bathroom 1
4 Daniel went back to the hallway.
5 Sandra moved to the garden.
6 Where is Daniel? hallway 4

The results are comparable (or superior) to those for the LSTM baseline provided in Weston et al.'s Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks given only 1000 samples and without any hyperparamater tuning.

Task Number	FB LSTM Baseline	Keras QA
QA1 - Single Supporting Fact	50	52.1
QA2 - Two Supporting Facts	20	37.0
QA3 - Three Supporting Facts	20	20.5
QA4 - Two Arg. Relations	61	62.9
QA5 - Three Arg. Relations	70	61.9
QA6 - Yes/No Questions	48	50.7
QA7 - Counting	49	78.9
QA8 - Lists/Sets	45	77.2
QA9 - Simple Negation	64	64.0
QA10 - Indefinite Knowledge	44	47.7
QA11 - Basic Coreference	72	74.9
QA12 - Conjunction	74	76.4
QA13 - Compound Coreference	94	94.4
QA14 - Time Reasoning	27	34.8
QA15 - Basic Deduction	21	32.4
QA16 - Basic Induction	23	50.6
QA17 - Positional Reasoning	51	49.1
QA18 - Size Reasoning	52	90.8
QA19 - Path Finding	8	9.0
QA20 - Agent's Motivations	91	90.7

For the resources related to the bAbI project, refer to the Facebook AI Research bAbI project page.

License

MIT License, as per LICENSE