elena-vm / warsawpublictransport

WarsawPublicTransport

Getting Started

1. Clone the repository with the following command git clone https://github.com/klepper24/WarsawPublicTransport.git GIT

2. Install Docker

3. Run docker-compose up airflow-init to initialize environment

4. Start all services with docker-compose up -d command

5. Open webserver which is available at http://localhost:8080

   • login: airflow
   • password: airflow

6. Configure api_key variable:

   • In order to get api_key go to this page https://api.um.warszawa.pl -> Logowanie -> Rejestracja konta and fill out the form
   • 
   • 

7. Configure connections

   • 
   • API connection:
     • 
   • Spark connection:
     • 
   • File Sensor connection:
     • 

8. Turn on relevant DAGs

Local development

1. Create local virtual envirtonment

   python3.8 -m venv venv  
   source venv/bin/activate

2. Install poetry (Instalation guide)

   curl -sSL https://install.python-poetry.org | python3 -
   poetry install

3. Install dependencies locally

pip install cryptography==36.0.2
poetry install

Cleaning-up the environment

To stop and delete containers, delete volumes with database data and download images, run docker-compose down --volumes --rmi all

CodeQuality

tox -e flake8
tox -e pylint