kpsychas / word_etymologist Goto Github PK

The output of NN is a sequence of classification labels, here True, False,
but it is unlikely if at all possible to have only a single or even just two letter in a row
classified as True/False and the rest of the letters being classified the opposite.

There is need of a function that can even be thought as an extra layer that makes
output "smoother".
A smoother output satisfies the following requirements.

• Should be as close as possible to the original output sequence
• Given a parameter n and a classifications sequence S,
  then for every label in S there is a subsequence of length
  n that contains the label and every label in the subsequence is
  the same.
• If length of subsequence is less than n, all labels in output sequence
  should be the same
• Assume that the latest labels have more weight.

The project currently skips words that are invalid without printing details about the Error.
Better error handling should include:

1. The error type
2. For all foreseen errors make a decision of whether to interrupt program and print a trace or continue training if it is not fatal.

Add an option to train the network over the whole dataset one or more times.
Currently words are picked randomly and some words may never be picked.

The name of annotated word file should be a parameter of train.py script.
Optional extra features

1. Multiple input files can be loaded in the same script and merged into one structure.
2. Duplicate words are removed.

The following project can be a starting point.

https://github.com/PySimpleGUI/PySimpleGUI

This can be a separate script that prompts used to input a word and optionally what its expected output is.
The model should respond with the predicted output.

Bonus:
Keep a history of training inputs and allow user to reuse them.

Currently words are offered to the network one by one. A better framework would allow multiple words to be trained at the same time. To do that we either need to pad all words in a batch to have the same size, or group words by length and sample words from one group at a time to make a batch.

Even though I have not found explicit support from libraries, such a model should be possible.
A place to start is the following publication
https://www.researchgate.net/publication/3316656_Bidirectional_recurrent_neural_networks

Is a continuation of #7 and should better be implemented after #3.

Minimum requirement is to be able to train and evaluate a model without extra arguments.
That means the following sequence of commands should run without errors.

python train.py --program 1
python train.py --program 2