jasonluo-tw / llm_rag

Simple demo for RAG

Simple demo of the RAG(Retrieval Augmented Generation)

1. Use Bert-based Model developed by ckiplab to extract text embeddings.
2. Use ResNet50 or EfficientNet_B7 to get the image embeddings.
3. Create vector database through Milvus to store embeddings including texts and images embeddings.
4. Use similarity search to find the similar images or QA texts.

How to run it

• At first, download docker compose yaml file

  wget https://github.com/milvus-io/milvus/releases/download/v2.3.2/milvus-standalone-docker-compose.yml -O docker-compose.yml

• To run Milvus, execute the command below

  sudo docker-compose up -d

• Check if all the containers are activated. There will be three docker containers running(milvus-etcd, milvus-minio, milvus-standalone)

  sudo docker compose ps

• Download the demo data found on the Internet(include texts & images)

  mkdir data
  cd data
  ## Get text data
  wget https://github.com/A-baoYang/finetune-with-openai/raw/main/example_data/faq.jsonl
  
  ## Get image data (manual download from the web-site or use the kaggle API)
  kaggle datasets download -d jehanbhathena/weather-dataset
  unzip archive.zip

• Run create_img_db.py to create the collection for images

  python create_img_db.py

• Run create_QA_db.py to create the collection for texts

  python create_QA_db.py

• Run img_search.py or QA_text_search.py to do the similarity search

Ref.

• Milvus
• faq jsonl data
• Bert-based model from ckiplab
• weather image recognition
• Towhee - reverse image search