๐ข๐๐ฒ๐ฟ๐๐ถ๐ฒ๐
Web scrapping of a web page consisting of new articles snippets of various different categories was done to extract snippet, title and category. Only three categories were selected books, film and music. This data was stored in a text file and further processing was done.
๐ง๐ฎ๐๐ธ ๐ฐ๐ผ๐บ๐ฝ๐น๐ฒ๐๐ฒ๐ฑ
โข Text mining
Beautiful soup 4 was used to parse through all the pages of the given website and it was stored in a text file on the local system.
โข Text pre-processing
Stop words such as '๐ต๐ฉ๐ฆ,๐ข๐ฏ, ๐ข๐ฏ๐ฅ,๐ข,๐ฐ๐ง,๐ง๐ฐ๐ณ' were removed, words which had frequency of less than 5 in the corpus were removed. Along with this lemmatization was used to understand the common root words.
โข Binary classification
At a time only two categories were given to a linear model. This model was trained with randomly split training data set and then tested with the test data. The check the results accuracy and confusion matrix were displayed. For SDGC classifier 90% accuracy was obtained.
โข Multi-class classification
For multi-class all the three categories were given to the model and similar process as above was followed. An accuracy of 88% was obtained when SDGC classifier was used whereas logistic regression model only gave an accuracy of 86%