jeremyarancio / rpg_assistant

Live amazing stories with this role play gaming assistant

Fireball dataset

Extraction

The dataset is massive once fully extracted.

wget -O fireball.tar.gz https://datasets.mechanus.zhu.codes/fireball-anonymized-nov-28-2022-kfdjl.tar.gz
tar -zxvf fireball.tar.gz && rm fireball.tar.gz
find anonymized/data -name '*.gz' -exec gzip -d {} \; #same for filtered

EDA

rpg_assistant's Issues

Train model

Train model using Sagemaker

Switch libraries management to pip / setup.py / requirements.txt

Poetry seems really buggy and not applicable to all deployment
Go back to requirements.txt with setup
Check MadewithML website

Inputs.to_cuda() in inference.py in the artifact

Some features for text generation doesn't work if inputs is not to_cuda()

In addition, build a .sh file to apply changes to the artifact inference.py in a CI/CD way

Prepare data

Build app

The app should:

• Propose an overall story to the game master
• Based on the player's actions, propose 3 scenarios
• Keep in memory what happened before

Define initial product architecture

• What does the product do?
• DB ? Features?
• What is the main feature?

Fireball dvc pipeline

Define the pipeline to get Fireball dataset prepared for LLM fine-tuning v1

Prepare repo for reproducibility

• Setup.py
• Requirements
• flake
• blake
• isort
• mkdocs
• makefile