Tech News is a Python-based data scraping project that uses Beautiful Soup to perform web scraping on Trybe's blog and fetch technology news and articles, storing them in a non-sequential MongoDB database.
✨ Show Tecnologies
- PYTHON 3
- MONGODB
- PYMONGO
- PYTEST
- BEAUTIFUL SOUP
- FLAKE
- Collect news and articles from Trybe's blog using web scraping and store them in a MongoDB database
- Search news by title
- Search news by date
- Browse news by category
- List the top 5 categories with the most news
To run the project locally, follow the steps below.
- Check that your machine has the minimum configuration for the project to run;
- Python 3;
- Docker;
- Docker-compose version equal to or greater than
1.29.2.
- Clone the repository;
git clone https://github.com/RuhamLeal/tech_news_scraper.git
- Navigate to the root of the project;
cd ./tech_news_scraper
- Create and activate the virtual environment.
python3 -m venv .venv
source .venv/bin/activate
- Install dependencies in the virtual environment.
python3 -m pip install -r dev-requirements.txt
- If you don't have MongoDB installed locally, launch it via Docker.
docker-compose up -d mongodb
7.Run the following command to access the menu.
tech-news-analyzer
Ruham Leal
Email: [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent