bevacqua / fuzzysearch

:crystal_ball: Tiny and blazing-fast fuzzy search in JavaScript

fuzzysearch

Tiny and blazing-fast fuzzy search in JavaScript

Fuzzy searching allows for flexibly matching a string with partial input, useful for filtering data very quickly based on lightweight user input.

Demo

To see fuzzysearch in action, head over to bevacqua.github.io/horsey, which is a demo of an autocomplete component that uses fuzzysearch to filter out results based on user input.

Install

From npm

npm install --save fuzzysearch

fuzzysearch(needle, haystack)

Returns true if needle matches haystack using a fuzzy-searching algorithm. Note that this program doesn't implement levenshtein distance, but rather a simplified version where there's no approximation. The method will return true only if each character in the needle can be found in the haystack and occurs after the preceding matches.

fuzzysearch('twl', 'cartwheel') // <- true
fuzzysearch('cart', 'cartwheel') // <- true
fuzzysearch('cw', 'cartwheel') // <- true
fuzzysearch('ee', 'cartwheel') // <- true
fuzzysearch('art', 'cartwheel') // <- true
fuzzysearch('eeel', 'cartwheel') // <- false
fuzzysearch('dog', 'cartwheel') // <- false

An exciting application for this kind of algorithm is to filter options from an autocomplete menu, check out horsey for an example on how that might look like.

But! RegExps...!

_{The current implementation uses the algorithm suggested by Mr. Aleph, a crazy russian compiler engineer working at V8.}

License

MIT

fuzzysearch's Issues

levenshtein algorithm

What would be cool is to add the levenshtein algorithm to sort these results.

Circuit breaker improvement

You can return a little earlier when there is no match by replacing the second breaker with:

if (qlen === tlen) {
   return query === text;
}

Since if the lengths are equal but the query does not equal the text, there is no way the query will match the text.

needle === haystack ?

I don't get this part:

if (nlen === hlen) {
    return needle === haystack;
  }

this means:
"apple" === "appel" false

Shouldn't this example return true?

Why is nlen === hlen important ?

Support unicode fuzzy search

I've just took a look at the source code and I saw str.charCodeAt is used instead of str.codePointAt. So fuzzy searching unicode characters (multibytes characters for example) is probably not supported by this algorithm.

Case sensitive

Any way to incorporate case insensitive option/flag?

Regex seem to be almost as performant

From my tests on a ~400 kb dataset, the the mraleph's approach was barely faster than regexps, not several orders of magnitude as the graph in the README shows.

As for the jsperf test results, then that might be due to toReg and escapeRegExp. That should be done once, not on every iteration.

Edit: very odd. When the needle is english the regex doesn't perform well. My initial tests were with an Arabic needle.

Edit two: so regexes turned out to be quite close in performance, but it depends on the needle and query.

indexOf is just as fast

I realize the so-called "high resolution" timers in JS are fucked with due to security reasons, but see below:

var timeOrigin = performance.now() ;fuzzysearch("wrt", "wrtten") ; var timeOrigin2 = performance.now()

timeOrigin2 - timeOrigin
0.0050000089686363935

var timeOrigin = performance.now() ;"wrtten".indexOf("wrt") ; var timeOrigin2 = performance.now()

timeOrigin2 - timeOrigin
0.0050000089686363935

Test framework

Why not use mocha with should instead of tap?

Where can I find the chat with Mr. Aleph?

Do you mind pointing me to the issue/PR/discussion you had with Mr. Aleph where he suggested his take?

Different from indexOf?

I might be misunderstanding something here, but how is this different from:
haystack.indexOf(needle) >= 0

Is it faster?

Interesing, and I got one too:

Mine was in a functional one written in CirruScript(compiles with http://repo.cirru.org/script/):

= generateSearch $ \ (text query info index)
  -- "return if matches"
  if (is text query)
    do $ return $ object (:match false) (:start 0)
  -- "reurn false if text is using up"
  if (and (is text.length 0) (> query.length 0))
    do $ return $ object (:match false) (:start 0)
  -- "return true if query is using up"
  if (is query.length 0)
    do $ return info

  = nextIndex $ + index 1
  -- "first letter matches, keep going without first letters"
  if (is (. text 0) (. query 0))
    do
      if (< nextIndex info.start)
        do $ = info.start nextIndex
      return $ generateSearch (text.substr 1) (query.substr 1) info nextIndex
  -- "first letter does not match, but keep going without first letter of text"
  generateSearch (text.substr 1) query info nextIndex

= exports.fuzzyStart $ \ (list query)
  = result $ list.map $ \ (text)
    = info $ object (:start 10) (:match true) (:text text)
    generateSearch text query info 0
  -- "filter and sort"
  = result $ result.filter $ \ (item) item.match

  = result $ result.sort $ \ (a b)
    cond
      (< a.start b.start) -1
      (> a.start b.start) 1
      else 0

  return $ result.map $ \ (item) item.text

Support for a types file for use in Typescript

Any chance a type declaration file could be included?

Added to Semantic UI

Just wanted to let you know that I've added your fuzzy search algorithm to our search component

Code can be seen here

What about...

function fuzzysearch(query, data) {
    return (new RegExp(query.split('').join('.*'))).test(data);
}