- Use Bert-based Model developed by ckiplab to extract text embeddings.
- Use ResNet50 or EfficientNet_B7 to get the image embeddings.
- Create vector database through Milvus to store embeddings including texts and images embeddings.
- Use similarity search to find the similar images or QA texts.
- At first, download docker compose yaml file
wget https://github.com/milvus-io/milvus/releases/download/v2.3.2/milvus-standalone-docker-compose.yml -O docker-compose.yml
- To run Milvus, execute the command below
sudo docker-compose up -d
- Check if all the containers are activated. There will be three docker containers running(milvus-etcd, milvus-minio, milvus-standalone)
sudo docker compose ps
- Download the demo data found on the Internet(include texts & images)
mkdir data cd data ## Get text data wget https://github.com/A-baoYang/finetune-with-openai/raw/main/example_data/faq.jsonl ## Get image data (manual download from the web-site or use the kaggle API) kaggle datasets download -d jehanbhathena/weather-dataset unzip archive.zip
- Run create_img_db.py to create the collection for images
python create_img_db.py
- Run create_QA_db.py to create the collection for texts
python create_QA_db.py
- Run img_search.py or QA_text_search.py to do the similarity search
Ref.