adaptive-icl-labeler's Introduction

Adaptive In-Context-Labeler

Using In-Context Learning Capability, LMQL, and multi-armed bandit examplars optimization for fast labeling pipeline

Currently WIP, but feel free to try out.

Built with react, LMQL, fastap, replicate

Demo

First, run frontend and backend server via

cd frontend
npm run start
cd ../backend
uvicorn main:app --reload

Run local server or use replicate backend. ("replicate:charles-dyfis-net/llama-2-13b-hf--lmtp-8bit")

For local server, for example, run lmql serve-model meta-llama/Llama-2-13b-chat-hf --port 8010 --cuda --load_in_4bit True

For replicate backend, don't run anything, instead, set osenv variables including api token

export REPLICATE_API_TOKEN= ... # YOUR API TOKEN
export MODEL_ID="meta-llama/Llama-2-13b-chat-hf"
export ENDPOINT="replicate:charles-dyfis-net/llama-2-13b-hf--lmtp-8bit"

Use the app at http://localhost:3000/. You would have to have your data in backend/data.csv file, with item_id,text,processed_value,is_processed columns. You can use backend/seed.py as an example dataset creation.

TODOS:

Keep track of preference statistics per examplars, and use UCB based multi-armed bandit to select examplars
Add more flexibility for saving - loading current state, maybe like json file or something
Upload data CSV and have option to do everything in browser maybe?
Able to edit examplars after creation
Abstract out LMQL so that non-tech people can use it
Put nice documentations

Recommend Projects