anwaraliahmad / callcenter

AI appointment scheduling call center tech demo

AI Appointment Scheduling Call Center Demo

A tech demo for scheduling an appointment by speaking with an AI healthcare call center.

About

This project creates a server instantiating a publicly exposed AI call center to schedule doctor appointments. Upon calling the number, the patient will be speaking with a conversational AI agent (by default "Rachel") which will:

1. Guide them through sharing relevant healthcare intake info for scheduling
2. Offering example availabilities (time, place, doctor)
3. Confirming appointment based on availability selection
4. Sending text to confirm said appointment

Stack

• Local Python FastAPI RESTful server with ngrok tunnel for exposing it
• AI vocoding and telephony managed by vocode.dev, which orchestrates with the following services:
  • Transcribing (caller voice to text) by Deepgram
  • Synthesizing (AI text to voice) by ElevenLabs
  • Agent (the AI model itself) by OpenAI
  • Telephony (in/outbound calling) by Twilio
• Redis for in-memory storage

Documentation

While there are already choice links to tech sites and docs throughout this README, the following are some more specific links which I found to be particularly useful.

Vocode

• Vocode python quickstart
• Vocode action agents
• Streaming examples
• Application examples
• App examples hosted on Replit
• Sourcecode for AI agents
• Sourcecode for AI actions
• Searching the Vocode-python repo

ngrok

• Getting started

OpenAI

• Make API key

Twilio

• Sending SMS message with Python

Prerequisites & Assumptions

• Python 3.8+
• Audio processing: ffmpeg
• Python dep management: Poetry
• Python virtual environment: venv
• Memory: Redis
• Server: ngrok
• Containerization (optional): Docker
• API Tokens (See Installation on where to put the details):
  • Deepgram
  • OpenAI
  • ElevenLabs
  • Twilio (must have phone number)

Project Structure

.
├── .env.template # Environment template file
├── .gitignore # Which folders and files git will not track
├── README.md # This file
├── callcenter # Source folder for additional Python modules
│   ├── action_factory.py # Factory to handle creating custom actions in subfolder below
│   └── actions # Custom actions for the AI agent (anything we want it to do beyond talking)
│       ├── best_times.py # Return best appointment availabilities to user (info required)
│       └── send_text.py # Final step (info required + appointment accepted) of sending confirmation text
├── config.yaml # Project-specific values (right now only used for constants)
├── docker-compose.yml # Docker configuration
├── Dockerfile # The Docker container "recipe"
├── main.py # The entry point and core logic of the server application
├── poetry.lock # List of the installed versions of Poetry dependencies
├── pyproject.toml # Defines project information and poetry dependencies
└── requirements.txt # Pip-friendly list of dependencies 

Installation & Running

Based on the very useful vocode Python docs If you find yourself wanting more details check that out.

1. Clone / download this repo and cd into it
2. Create environment file out of template then fill with corresponding values with cp .env.template .env
3. Set up tunneling with ngrok http 3000

• Be sure to configure ngrok
• This is crucial for communicating with the outside world and services, especially webhooks
• In .env change BASE_URL to <yoururl>.ngrok.app generated by ngrok
• Be sure to add a webhook to your Twilio number with the url https://<yoururl>.ngrok.app/inbound_call to enable our server defined in main.py to handle incoming calls with its inbound_call endpoint

4. Open another terminal window, setup a virtual environment: python3 -m venv env; source env/bin/activate
5. Install dependencies with poetry install

• Make sure you install Poetry

6. Launch Redis: brew services start redis
7. Run the actual server: poetry run uvicorn main:app --port 3000
8. Test it out by calling your Twilio number!

Final Notes

Tech Eval

Pros

• Vocode easily connects with services for transcription, synthesizing, and AI agents
• ActionAgents (used here) are very powerful, a docstring can tell ChatGPT what action to trigger and what to pass it
• Actions and Agents are highly customizable

Cons

• Open source library nascent, and there's a lack of documentation for especially newer and more rapidly changing features such as actions
• There were some very ad-hoc changes required to get certain custom actions to work, such as needing to modify the ActionFactory in original the Python module file itself (dev confirmed this is currently the way to do this)

Looking Forward

• Using redis and kubernetes on cloud deployments not only great for data managament but scalability of service
  • SortedSets can make use cases for scheduling faster (e.g. pulling best times)
• For HIPPA compliance and competitive edge, self-managed AI agents can be used instead (See the list here)
  • If CICD and resource usage can be optimized, a custom LLM agent such as Llama can be trained and used
• Use of langchain could be very powerful to handle autonomous actions such as contacting insurance to validate information (think of the applications beyond scheduling)