evanlab02 / lyzer-etl

Hi, my name is Evan 👋

I am very passionate about making an impact with technology and solving problems. I am interested in Backend development, frontend development, Dev Ops, Cloud Solutions and Data Science. Currently an associate software engineer at NTT Ltd.

• 🔭 I’m currently working at NTT Ltd. where we are working with Python and React. Also working on a ShoppingList application (check out my repositories)
• 🌱 I’m currently learning more about front-end development and advanced python concepts.
• 📫 How to reach me: [email protected]
• 💡 Fun fact: I love surfing but I am not very fond of the beach and sand.

Reach out to me

Languages and Tools

lyzer-etl's Issues

Create TUI for App

Description

It can be convenient to rather have some UI to do hydration instead of just a CLI command. We will use a TUI framework to create a terminal interface that will be able to hydrate data.

Acceptance Criteria

• Create TUI that can hydrate schedules into the database.
• Add CLI command that will start Terminal User Interface.

Identify data structure for driver results/standings as well as constructor results/standings (Race Results Only)

Create notifications using ntfy.sh for events

Create setup function

Description

We need to create a function that sets up the app with all keys and secrets that will be required in a local config file.

The config file should be stored at {home directory}/.lyzer/hydrate-config.json

Acceptance Criteria

• Setup retrieves MongoDB connection string from user.
• Setup saves MongoDB connection string in local config file.
• Setup ensures MongoDB connection string is given and is valid.

Create Web UI

Description

We would like to also have the chance to use a web interface for hydration if preferred. This should be easy to implement using Streamlit.

Acceptance Criteria

• Create Web UI for ETL.
• Add command that starts Web UI.

Setup basic project structure

• Setup basic project structure
• Linting
• Testing
• Building
• CI (Linting, Testing and Building)
• CD (release-please)

Create config command

Description

We need a config command that can adjust config settings

Acceptance Criteria

• Config command should be able to validate and update the MongoDB connection string with given string.

Create Hydrate Schedule Command using Typer

Description

We would like to be able to hydrate a years schedule using this app.

The command would look something like the following.

Lyzer-Hydrate load schedule 2023

Acceptance Criteria

• Create sub commands file for hydrate commands.
• Command should retrieve data from ergast for given year.
• Command should refine data for our database.
• Command should put refined data into the database.
• Command should reject invalid years.
• (Optional) Print data that was inserted using table print of some sort

Setup Auto Updater using Github API

Description

We would like to have our app auto-update when new releases are released to Github, this will download the assets/artifacts so that we can have easier time updating our app.

Acceptance Criteria

• Downloads new release artifacts when releases are made.
• Attempts to, where possible to replace the current binary/executable with the new one.
• Restrict users from performing any commands if they are not on the latest version.