sandrobarna / image_search_demo

Demo project showing semantic text2image search using vector database

Image Search using Text

Description

This is demo project showcasing image similarity search using text queries. Project uses CLIP model for text/image embeddings Qdrant vector database for KNN search.

Architecture

The app contains following components:

• Qdrant service (runs locally)
• Similarity search backend (see backend folder in repo) that computes embeddings and communicates with qdrant service
• FastAPI based REST API and HTML frontend
  • For demo purposes, functionality is limited to K-NN search only (adding/editing existing images isn't supported from web)

How to Use

1. Clone the Repository:

   git clone https://github.com/sandrobarna/image_search_demo.git
   

2. Download Data:

Embedding model weights, test images as well as Qdrant's collection pre-populated with those image vectors need to be downloaded and mapped to respective docker volumes. Download it from HERE.

3. Spin-up services:

   There is a docker-compose file in repo which spins up Qdrant service (default port 6333), jupyter notebook (default port 1235) for playing with exploratory data analysis (backend/notebooks/exploratory_analysis.ipynb) and FastAPI server (default port 2222) for using HTML frontend.
   
   

   You need to map embedding_model and images subfolders (from the data downloaded in step 2) to /searchapp_data docker volume as shown below (in docker-compose.yaml).

    volumes:
      - ./data/embedding_model:/searchapp_data/embedding_model # embedding model weights
      - ./data/images:/searchapp_data/images # folder containing images
   

   Now map Qdrant's storage folder qdrant_storage (also from the data downloaded in step 2)

   volumes:
      - ./data/qdrant_storage:/qdrant/storage # qdrant storage (already contains demo collection pre-populated with images)
   

   
   Modify default ports for jupyter or webapp if you wish in docker-compose.yaml
   
   Once all set, run following to fire up:

   docker-compose up

4. Access Web App:

   Open web browser and go to one of the following, as you wish:

   • localhost:2222 to try out demo webapp
   • localhost:1235 to go to Jupyter. You can play with backend/notebooks/exploratory_analysis.ipynb in order to see some good/bad query examples as well as some data stats. Jupyter secret token can be copied from docker-compose logs printing in terminal
   • localhost:6333/dashboard to go to Qdrant's dashboard