3ayazaya / habrrssparser

Parser for Habr.com via RSS and write them to PostgreSQL database

habrRSSparser

Parse and read simple

habrRSSparser is a python script for parsing habs from habr.com via RSS and write them to PostgreSQL database.

Install app

habrRSSparser requires docker and docker-compose to run.

Clone repo

git clone https://github.com/3ayazaya/habrRSSparser
cd habrRSSparser

Create .env file

touch app/.env

Configure .env file

vim app/.env

Paste and configure

DB_USER='<YOUR_DATABASE_USER_FOR_CONNECTION>'
DATABASE='<YOUR_DATABASE_NAME_FOR_CONNECTION>'
DB_PASSWORD='<YOUR_DATABASE_USER_PASSWORD_FOR_CONNECTION>'

Build app

Set env

set -a
source app/.env

Build app with docker-compose

docker-compose build

Run app

Running app with docker-compose

docker-compose up -d

Logs

Logs are stored in app/logs folder

Modules

habrRSSparser is currently use the following plugins. Instructions on how to use them are linked below.

Module	README
colorama	pypi
feedparser	pypi
loguru	GitHub
psycopg2	psycopg
time	docs.python
python-dotenv	pypi