spect3r-69 / hungryhippo

🦛 scrapes websites and generates rss feeds

HungryHippo

To a hungry hippo, everything is a feed

HungryHippo generates RSS/ATOM/JSON feeds from regularly-updating public sites.

Examples

Site	Sample Link	HungryHippo Link
Facebook	facebook.com/smbccomics	/facebook/smbccomics
Twitter	twitter.com/onedevloperarmy	/twitter/onedevloperarmy
Instagram	instagram.com/nathanwpylestrangeplanet	/instagram/nathanwpylestrangeplanet
UCL News	ucl.ac.uk/news	/individual-site/ucl.ac.uk/news/?url=https://search2.ucl.ac.uk/s/search.json?collection=drupal-push-news-news&meta_UclCommunicationType=%22top+stories%22
Telegraph	telegraph.co.uk/opinion	/individual-site/telegraph.co.uk/?url=https://www.telegraph.co.uk/opinion/
UCL Laws News	ucl.ac.uk/laws/news	/individual-site/ucl.ac.uk/news/?url=https://cms-feed.ucl.ac.uk/s/search.json?collection=drupal-laws-new%26&meta_UclOrgUnit=%22UCL Faculty of Laws%22&title=UCL%20Laws%20News
UCL UCU News	ucl.ac.uk/ucu/news	/individual-site/ucl.ac.uk/news/?url=https://cms-feed.ucl.ac.uk/s/search.json?collection=drupal-professional-services-news%26meta_UclOrgUnit="UCL UCU"&title=UCL UCU
Singapore Law Blog	singaporelawblog.sg	/individual-site/singaporelawblog.sg
The Edge (SG)	theedgesingapore.com	/individual-site/theedgesingapore.com
PDPC (SG)	pdpc.gov.sg	/individual-site/pdpc.gov.sg/undertaking
GitHub Code Search	github.com/search	/github?q="xkcd.com"

Pop the HungryHippo link into a RSS reader like Feedly, BazQux, or RSSOwl. Or you can use IE.

You can get a list of supported sites by visiting /individual-site.

Running HungryHippo

With docker-compose:

  docker build . -t hungryhippo
  docker run -p 3000:3000 hungryhippo

Without docker:

  npm ci
  npm run start:dev

GitHub

Optionally, provide the GITHUB_OAUTH_TOKEN environment variable to enable the GitHub search RSS feeds.

Development

I may have taken the Hungry Hippo theme a bit too far. In short:

• Muncher: scrapes websites
• Digestor: turns scraped items into feeds

Credits

Photo taken by David Goehring