schwerdt / takingsearchforaspin Goto Github PK

In this project, I am trying to implement a search method.  
I made up a few articles about animals that the program 
searches, given the string that the user provides.  

Each article added for searching must go in to the file 
articlelist  (one article per line).  

The code works by constructing a word vector for the search string.  Then 
the code runs through the articles and computes a word vector for each
article. 

The words in the word vector consist of the words in the search string.
Then the word vectors for each article have word counts in that particular
article for the words in the search string.

Ex.
search is:  "cats are furry"
so the words in the word vector are 'cats', 'are', 'furry'
The counts of those words in each article are tracked.
If the article has the word 'cats' 2x, 'are' 3x and 'furry' 1x, the word
vector is:
[2, 3, 1]

This vector can be normalized.

The simple_search.py code is very simple and uses minimum Euclidian 
distance between articles and the search string.  In search.py, I was 
testing the Inverse Document Frequency measure on the search.  This causes 
words that appear in a lot of articles to be given less importance 
(through a smaller weighting factor).  One example of this is the 
word 'are'.

I plan to test different methods for finding the best match and then work on adding different article sources.