matthewcylau / python-fear-greed-scraper

A Python web scraper which scrapes the Fear and Greed index

Python Fear and Greed Scraper

A Python web scraper which scrapes the Fear and Greed index

The list of repositories are as follow:

• Python Flask API repository here
• React with Vite client repository here
• Scraper and GCP infrastructure repository here

Run/build app locally

• Run app on host machine:

virtualenv -p /usr/bin/python3 venv
source venv/bin/activate
pip3 install -r requirements.txt
python3 manage.py
deactivate

• Run app as container:

docker compose up --build

Install new packages

pip3 install boto
pip3 freeze > requirements.txt

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

python-fear-greed-scraper's Issues

Add Terraform to run Cloud Run job on a schedule

GCP role assignment

• https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#google_project_iam_member
• https://cloud.google.com/run/docs/reference/iam/roles

Send email notification

https://docs.sendgrid.com/for-developers/sending-email/quickstart-python

Log out time taken for scraping.

Create, or replace Cloud Run Job

https://stackoverflow.com/questions/75446719/gcp-create-or-replace-cloud-run-job