gustavz / whatsbot

python flask app serving as webhook for whatsapp business accounts making prompts to openai api

WhatsBot

How DALL-E 2 imagines the project

Getting-Started

To create your own WhatsApp chatbot you all you need to do is:

1. Creata an OpenAI API account and generate an API key
2. Create a Meta developer account
3. Create a Whatsapp Business app and generate a Token
4. Remix the app on Glitch and set environmental variables
5. Link your glitch app as webhook to your WhatsApp app
6. Last but not least: star this project on GitHub ❤️

TL;DR

How ChatGPT describes the project

This program is a webhook implemented in python and flask. It receives incoming messages from WhatsApp, processes and responds with the help of OpenAI API. The messages can be either text or audio, and the response is generated with OpenAI's GPT-3.5 model. The critical functions of the program include handling incoming messages of different types, converting audio files to text and sending responses back to the user.

Learnables

• Python programming language
• OpenAI API
• WhatsApp Cloud API
• Webhooks
• Flask web framework
• PyDub for audio manipulation
• SpeechRecognition for speech to text conversion
• Soundfile for working with audio files

Code Deep-Dive

The program consists of several functions to handle different aspects of the process.

As first step verify verifies that the webhook connection is trustworthy.
It does so by checking that the verfifcation token and correct mode are set in the request.

def verify(request):
    # Parse params from the webhook verification request
    mode = request.args.get("hub.mode")
    token = request.args.get("hub.verify_token")
    challenge = request.args.get("hub.challenge")
    # Check if a token and mode were sent
    if mode and token:
        # Check the mode and token sent are correct
        if mode == "subscribe" and token == verify_token:
            # Respond with 200 OK and challenge token from the request
            print("WEBHOOK_VERIFIED")
            return challenge, 200
        else:
            # Responds with '403 Forbidden' if verify tokens do not match
            print("VERIFICATION_FAILED")
            return jsonify({"status": "error", "message": "Verification failed"}), 403
    else:
        # Responds with '400 Bad Request' if verify tokens do not match
        print("MISSING_PARAMETER")
        return jsonify({"status": "error", "message": "Missing parameters"}), 400

handle_message processes incoming WhatsApp messages and does error handling

def handle_message(request):
    # Parse Request body in json format
    body = request.get_json()
    print(f"request body: {body}")

    try:
        # info on WhatsApp text message payload:
        # https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples#text-messages
        if body.get("object"):
            if (
                body.get("entry")
                and body["entry"][0].get("changes")
                and body["entry"][0]["changes"][0].get("value")
                and body["entry"][0]["changes"][0]["value"].get("messages")
                and body["entry"][0]["changes"][0]["value"]["messages"][0]
            ):
                handle_whatsapp_message(body)
            return jsonify({"status": "ok"}), 200
        else:
            # if the request is not a WhatsApp API event, return an error
            return (
                jsonify({"status": "error", "message": "Not a WhatsApp API event"}),
                404,
            )
    # catch all other errors and return an internal server error
    except Exception as e:
        print(f"unknown error: {e}")
        return jsonify({"status": "error", "message": str(e)}), 500

handle_whatsapp_message is the main function that handles incoming messages of different types, and based on the type of the message, it calls the relevant function.

def handle_whatsapp_message(body):
    message = body["entry"][0]["changes"][0]["value"]["messages"][0]
    if message["type"] == "text":
        message_body = message["text"]["body"]
    elif message["type"] == "audio":
        audio_id = message["audio"]["id"]
        message_body = handle_audio_message(audio_id)
    response = make_openai_request(message_body, message["from"])
    send_whatsapp_message(body, response)

For example, if the message is an audio file, it calls handle_audio_message to convert the file to text.

def handle_audio_message(audio_id):
    audio_url = get_media_url(audio_id)
    audio_bytes = download_media_file(audio_url)
    audio_data = convert_audio_bytes(audio_bytes)
    audio_text = recognize_audio(audio_data)
    message = (
      "Please summarize the following message in its original language "
      f"as a list of bullet-points: {audio_text}"
    )
    return message

convert_audio_bytes is another critical function that converts the audio file into a format that can be processed by the speech_recognition library.
The function first converts the OGG file into WAV format and then converts it into an AudioData object.

def convert_audio_bytes(audio_bytes):
    ogg_audio = pydub.AudioSegment.from_ogg(io.BytesIO(audio_bytes))
    ogg_audio = ogg_audio.set_sample_width(4)
    wav_bytes = ogg_audio.export(format="wav").read()
    audio_data, sample_rate = sf.read(io.BytesIO(wav_bytes), dtype="int32")
    sample_width = audio_data.dtype.itemsize
    print(f"audio sample_rate:{sample_rate}, sample_width:{sample_width}")
    audio = sr.AudioData(audio_data, sample_rate, sample_width)
    return audio

The make_openai_request function makes a request to the OpenAI API and generates a response based on the user's message and the previous conversation log.
It uses the openai.ChatCompletion.create method to generate the response.

def make_openai_request(message, from_number):
    try:
        message_log = update_message_log(message, from_number, "user")
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=message_log,
            temperature=0.7,
        )
        response_message = response.choices[0].message.content
        print(f"openai response: {response_message}")
        update_message_log(response_message, from_number, "assistant")
    except Exception as e:
        print(f"openai error: {e}")
        response_message = "Sorry, the OpenAI API is currently overloaded or offline. Please try again later."
        remove_last_message_from_log(from_number)
    return response_message

The send_whatsapp_message function creates the Cloud API request sends the response back to the user via WhatsApp.

def send_whatsapp_message(body, message):
    value = body["entry"][0]["changes"][0]["value"]
    phone_number_id = value["metadata"]["phone_number_id"]
    from_number = value["messages"][0]["from"]
    headers = {
        "Authorization": f"Bearer {whatsapp_token}",
        "Content-Type": "application/json",
    }
    url = "https://graph.facebook.com/v15.0/" + phone_number_id + "/messages"
    data = {
        "messaging_product": "whatsapp",
        "to": from_number,
        "type": "text",
        "text": {"body": message},
    }
    response = requests.post(url, json=data, headers=headers)
    print(f"whatsapp message response: {response.json()}")
    response.raise_for_status()

whatsbot's Issues

Strange Problem

Hey bro, congrats for sharing this awesome work! I fork it on glitch to build my own version adding things like:

• User can send a news link and receive a summarized version.
• User can send an audio and get two responses: full and summary.

I don't know if you have the same problem, it's seems not be related to the code, but I think maybe I'm missing something about whatsapp API. The thing is, it's keep sending me the same old messges, again and again, did you got anything like that?

Question: I'm send two messages for response: full transcript and summary, is it a problem? I should send only one?

I'm googling a lot about it but didn't find anything.

S.O.S! haha