Extract text from Html about spider

[j-mendez]

Hi @MihirModi1421 take a look at this example here to extract content as text https://github.com/seoderapp/seoder/blob/main/seoder_lib/src/packages/spider/fs.rs#L171C6-L171C6.

The main gist is to use Html::parse_document(&my_html) and element.text().map(|s| s.chars()).flatten().collect() for the element nodes to get the text content. You can use the selector crate to use CSS selectors to get exactly what you want or just iterate over all the nodes of Html::parse_document.