oted / gsi-scraper

scraping the content

gsi-scraper

Worker for getting the content for GSI.

gsi-scraper's Issues

Add origin of content

Scrape vine

http://damn.com/

http://damn.com/

http://www.angryduck.com/

http://www.slideshare.net/

http://www.viralviralvideos.com/

http://www.viralviralvideos.com/

http://www.booooooom.com/

http://futurism.com/

http://gifsec.com/

http://gifsec.com/

scrape tumblr

http://www.hilariousgifs.com/

Scrape the theoatmeal

http://theoatmeal.com/

Create local db to avoid hitting api with duplications

http://www.cheezburger.com/

https://popkey.co/

Scrape vimeo

hell yahhhh

Make look over content

The content needs to be more interesting, for instance too many images does not make sense, there may be too many music videos etc...

Add way of cleaning up old stuff

9gag gifs does not work

http://www.huffingtonpost.co.uk/comedy/