spike: string matching algorithms about watchman

[adamdecaf]

FYI, we should look at how ES does Soundex and if we're normalizing the result similar to them.

https://www.elastic.co/guide/en/elasticsearch/guide/current/phonetic-matching.html (Should also check 6.x)

[adamdecaf]

Soundex is a phonetic algorithm for indexing names by sound, as pronounced in English.

(From: https://en.wikipedia.org/wiki/Soundex)

We will likely need to find non-english alternatives to use alongside.

Edit: From the paper directly below:

The Soundex and Metaphone algorithms are highly susceptible to missing matches where a name pair starts with a different letter (e.g. KAVANAGH, CAVANAGH and HAVANAH). This is especially true with names beginning with vowels - where typically many equivalents exist (e.g. EWELL, ULE, YOUL, WHEWELL, HEWEL), which should result in matches being reported.

[adamdecaf]

From https://waset.org/publications/8664/a-comparison-and-analysis-of-name-matching-algorithms-

Looks like the order of most accurate in that graph:

1. NYSIIS
2. Phonex
3. LIG2

I've also found some sample implementations:

• https://github.com/tdebatty/java-string-similarity/tree/master/src/main/java/info/debatty/java/stringsimilarity
• https://github.com/rockymadden/stringmetric/tree/master/core/src/main/scala/com/rockymadden/stringmetric/phonetic

[adamdecaf]

FYI, if we need to come back to this I think we should write custom ES scorers with specific string compare algorithms.