mbwolff / wvtm

A word vector text modulator

Word Vector Text Modulator

Based on a contribution to 2016 NaNoGenMo.

This repository contains the code and data necessary to modulate any text with word vectors derived from over 1,300 nineteenth-century French texts.

The following command will produce an interesting variation on the description of Charles Bovary's hat in the first chapter of Madame Bovary:

./modulateText.py maladroit adroit FlaubertBovary01.txt

Prerequisites

• Python 2.7
• treetagger
• Python modules:
  • aspell
  • pattern.fr
  • gensim
  • treetaggerwrapper

A quick explanation of what's under the hood

Vector space models of vocabularies offer promising techniques for exploring the semantic relationships between words. Using gensim to build such a model, the code presented here takes a pair of words (e.g. "homme" and "femme") and a given text as parameters to generate a new text based on computationally derived analogies. Each word in the original text is replaced by a word that is "most similar" to it according to the word pair. For instance, if "roi" is a word in the original text, it would be replaced like this:

>>> model.most_similar(positive=['femme', 'roi'], negative=['homme'], topn=1)
[(u'reine', 0.8085041046142578)]

The vector space model is able to complete the analogy: "reine" is to "femme" as "roi" is to "homme". Handling verb conjugations and adjective agreements computationally in French is tricky but the code produces a mostly readable text needing occasional grammatical polishing (a good exercise for students). The code can modulate any text against any pair of words. The quality of the result depends on the corpus for the vector space model and the word pair chosen by the user.