nicklondhe / mess

Multilingual Error based String Similarity measure

MESS

Multilingual Error based String Similarity measure

This repository is an implementation of the MESS algorithm and further changes therein, as introduced in

Londhe, Nikhil, et al. "MESS: A Multilingual Error based String Similarity measure for transliterated name variants." Proceedings of the 7th Forum for Information Retrieval Evaluation. ACM, 2015.

The MESS framework encodes transliteration errors as "weights" to be used in a distance measure. The library is pre-equipped with over 60 language pairs on transliteration models trained on Named Entities extracted from Wikipedia.

TODO:

1. Using MESS in your code
2. Adding pairs using external transliteration models
3. Solr plugin details etc.

Please cite the paper above if using this library in your work. Please email me ([email protected]) if you have questions.

mess's Issues

Process dictionaries to create alignment files

Evaluate AUC for measures on a common words dictionary

Need a class(es) to evaluate the PR response curve for a common words dictionary. The dictionary input is of the form:

DELIM

We need to evaluate how different measures react to such inputs by finding PR response curve to this input