meetrais / toastmaster-gen-ai-rag

Gen-AI specialized chatbot designed to answer fundamental questions about Toastmaster Program.

Toastmaster-Gen-AI - WORK IN PROGRESS

Introduction

Gen-AI specialized chatbot designed to answer fundamental questions about Toastmaster program. This project demonstrates how to develop retrieval augmented generation based chat-bot using Azure Open-AI service and Azure Cosmos-Db for Mongo-DB(VCore). Following are the main functionalities of this project.

1. Create database, collection and vector-index in Mongo-DB(VCore) cluster.
2. Create and save embeddings for PDF file in Mongo-DB(VCore) database.
3. Perform plain vector search for user query/question.
4. Perform chat-completion for user query/question.

High Level Architecture Diagram

Code Walkthrough/Setup

Azure Cosmos-DB for Mongo-DB(VCore) Cluster

Search for Azure Cosmos DB for MongoDB in the Azure portal and create cluster.

Folder/File Structure

Code overview

backend

1. createmongodbdbandindex.py - Run this file to create database, collection and vector-index in Mongo-DB(VCore) cluster.
2. embeddings.py - This file has below functions.
   generate_embeddings - Generate embeddings for the text using Azure Open AI.
   CreateAndSaveEmbeddingsForPDFFile - This function splits/chunks PDF file into pages, then creates embeddings and saves into MongoDB.
   perform_vector_search - This function performs vector search on MongoDB vector-index for the user query. Then returns vector-search result.
   perform_rag_vector_search - This function performs first vector-search and then RAG search by using OpenAI model defined for chat-completion.
3. app.py - This file contains HTTP get method getresponse which is called in frontend app for vector-search based chat-completion.
4. Dockerfile - This file is used to deploy backend app to docker.
5. Create .env file in backend folder. Then add below environment variables and initialize with your values.
6. requirements.txt - Contains list of Python libraries needed this program to run.

AZURE_OPEN_AI_ENDPOINT=""
AZURE_OPEN_AI_KEY=""
AZURE_COSMOSDB_MONGODB_USERNAME=""
AZURE_COSMOSDB_MONGODB_PASSWORD=""
AZURE_COSMOSDB_MONGODB_CLUSTER=""

frontend

1. app.py - To generate UI for Chat-Bot interface and call backend GET API for vector-search based chat-completion.
2. Dockerfile - This file is used to deploy frontend app to docker.
3. requirements.txt - Contains list of Python libraries needed this program to run.

Deployment

Docker Deployment

Once you tested your backend and frontend apps in your local its time to deploy them to Docker. These commands are for Windows OS.

Run below commands in the terminal of VS-Code/IDE to deploy and run backend app in Docker.
cd backend
docker build -t toastmaster-gen-ai-backend .
docker run --name toastmaster-gen-ai-backend -p 5000:5000 -d toastmaster-gen-ai-backend

Then run below commands to deploy and run frontend app in Docker.
cd..
cd frontend
docker build -t toastmaster-gen-ai-frontend .
docker run --name toastmaster-gen-ai-frontend -p 7860:7860 toastmaster-gen-ai-frontend

Both the above containers needs to be in the same Docker network so run below commands.
docker network create toastmaster-network
docker network connect toastmaster-network toastmaster-gen-ai-backend
docker network connect toastmaster-network toastmaster-gen-ai-frontend

Azure Deployment

I use VS Code for development and deployments. So following steps use VS-Code and its Docker and Azure plugins. But you can also do all those steps using Azure CLI commands.

Create Azure Container Registry

1. In Docker plugin click on your Azure Subscription and select Create Registry then follow steps to create ACR. You may have to login to Azure.
   

2. If you already have ACR in your Azure Subscription then you can connect to it as below.
   

Deploy Docker Image of Backend app to ACR

1. In Docker plugin inside IMAGES section locate your Image of backend app and then right-click and select Tag option.

2. Again right-click image and select Push option and follow steps to deploy your image to Azure Container Registry.

Create Azure Container App Environment

1. In Azure plugin click on + sign to create new Azure resource, select "Create Container Apps Environment" option and then follow the steps.

2. Once your Container Apps Environment is created, right-click on it and select "Create Container App" and follow the steps. Select "Container Registry" option and use 5000 for port. While doing this select ACR and Image of backend app.

   

Deploy Docker Image of frontend app to Azure Web-App

1. Before we deploy docker image of frontend app to Azure Web-App, make below change in app.py so frontend app will call backend app endpoint deployed in Azure Container app. Then re-deploy image of frontend app to Docker.
   

2. In Docker plugin, IMAGES section righ-click on docker image of frontend app and push to ACR. Just like we tagged and pushed docker image of backend app above.

3. After that you should be able to see docker image of frontend app in your ACR like below. Now right click on that image and select option "Deploy Image to Azure App Service.". Follow the steps to create Azure App Service and Service Plan. It will also deploy your docker image of frontend app to Azure App Service.

Showtime

If you followed all the steps properly you should be able to see result of Retrieval Augmented Generation based Chat-Bot as below.

If it doesnt work then reach out to me and I will be happy to help.