sayamalt / text-similarity-quantifier

Successfully developed a machine learning model for computing the similarity score between two text paragraphs taken as input from a webpage.

Text-Similarity-Quantifier

Objective

Establish an algorithm that can quantify the degree of similarity between the two text documents based on semantic similarity.

Semantic Textual Similarity (STS) assesses the degree to which two sentences are semantically equivalent to each other.

• 1 means highly similar
• 0 means highly dissimilar

Technologies Used

• Python
Libraries Used:
1. Numpy
2. Pandas
3. Seaborn
4. Matplotlib.pyplot
5. Joblib
6. warnings
7. string
8. Gensim Downloader
9. Sklearn
10. nltk
11. math
12. json
13. requests
• Flask
• Machine Learning
• Natural Language Processing

API Endpoint

The final algorithm should be exposed as a Server API Endpoint. In order to test this API, make sure you hit a request to the server to get the result as a response to the API. The request-response body should be in the following format:

Request body: {“text1”: ”nuclear body seeks new tech …....”, ”text2”: ”terror suspects face arrest ……”} Response body: {“similarity score”: 0.2 }

Note: “text1”, “text2”, and “similarity score” keys should be kept as it is, without any change.

Important aspect to consider

The given dataset does not contain any label. Therefore, can be treated as an unsupervised learning problem. However, this does not imply that supervised techniques/algorithms are not applicable. The candidate is free to use any technique.