iusaspb / demo-llm-tuning

Demo MLRun project for LLM Tuning and serving pipelines

MLOpsPedia - The MLOps Master Bot

This demo demonstrates how to fine tune a LLM and build an ML application: the MLOps master bot! We'll train falcon-7b on Iguazio's MLOps blogs and cover how easy it is to take a model and code from development to production. Even if its a big scary LLM model, MLRun will take care of the dirty work!

We will use:

• HuggingFace - as the main machine learning framework to get the model and tokenizer.
• DeepSpeed - as the distributed training framework.
• and MLRun - as the orchastraitor to operationalize it, moving it from development to production.

The demo contains a single notebook that covers the two main stages in every MLOps project:

• Training Pipeline Automation - Demonstrating how to get an existing model (falcon-7b) from HuggingFace's Transformers package and operationalize it through all of its life cycle phases: data collection, data ppreparation, training and evaluation, as a fully automated pipeline.
• Application Serving Pipeline - Showing how to productize the newly trained LLM as a serverless function.

You can find all the python source code under /src

Be sure to check out Yaron Haviv's video Deploying Hugging Face Models to Production at Scale with GPUs to get a walkthrough of a simillar demo.

---

Installation

This project can run in different development environments:

• Local computer (using PyCharm, VSCode, Jupyter, etc.)
• Inside GitHub Codespaces
• Other managed Jupyter environments

Install the code and mlrun client

To get started, fork this repo into your GitHub account and clone it into your development environment.

To install the package dependencies (not required in GitHub codespaces) use:

make install-requirements

If you prefer to use Conda use this instead (to create and configure a conda env):

make conda-env

Make sure you open the notebooks and select the mlrun conda environment

Install or connect to MLRun service/cluster

The MLRun service and computation can run locally (minimal setup) or over a remote Kubernetes environment.

If your development environment support docker and have enough CPU resources run:

make mlrun-docker

MLRun UI can be viewed in: http://localhost:8060

If your environment is minimal, run mlrun as a process (no UI):

[conda activate mlrun &&] make mlrun-api

For MLRun to run properly you should set your client environment, this is not required when using codespaces, the mlrun conda environment, or iguazio managed notebooks.

Your environment should include MLRUN_ENV_FILE=<absolute path to the ./mlrun.env file> (point to the mlrun .env file in this repo), see mlrun client setup instructions for details.

Note: You can also use a remote MLRun service (over Kubernetes), instead of starting a local mlrun, edit the mlrun.env and specify its address and credentials