nerds-amp-mods / hackernewsclone Goto Github PK

Request

GET /news
Authorization: Bearer <token>

Response

200 OK
{ 
    "articles": [
                      {  "title"       : '...',
                         "source"  : '....',
                         "topic"     :'....',
                          "url"        :'....'
                       },
                      { ... },
                  ],
    "next": "url"
}
---
404 Not Found

Tech

Finance

• Moneycontrol https://github.com/Web-Scrapers/Money-Control also they might have a api aswell cant find clear doc though

• CNBC 18 is another great source for this aswell

Startups
Meetups/Hackathons

• Meetup.com api
• Twitter hashtags added to track events stuff like hacktoberfest

jobs
Ask

Why this project?

Because we were discussing about the resourcefulness of hackernews. No doubt, it's fantastic. But the news is from all over world. We thought it might be a learning opportunity making a localized version of it.

What's in making a clone of something that's already there?

• Understand and implement large scale systems.
• Learn to write structured apis and structured code.
  It's Hacktoberfest folks!

What all can we do?

Categories for News

• Tech
• Finance
• Startups
• Meetups/Hackathons
• jobs
• Ask

Rss feeds can be extracted for sties that dont have api's.There are couple of third party api's out there which help fetching data from rss feeds. Although some sites cleary state they dont allow rss feeds for third party publishing. This is something that needs to be explored bit more.

Add an Asyncio scripts that runs all scrapers in intervals to periodically update the db with news articles.

Decide on a workflow mechanism to run async scripts.
Whether it should be a client that the scraper should call or one single script that runs all the scrapers at once is up to you.