svpino / alloy-voice-assistant

Sample AI assistant

You need an OPENAI_API_KEY and a GOOGLE_API_KEY to run this code. Store them in a .env file in the root directory of the project, or set them as environment variables.

If you are running the code on Apple Silicon, run the following command:

$ brew install portaudio

Create a virtual environment, update pip, and install the required packages:

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install -U pip
$ pip install -r requirements.txt

Run the assistant:

$ python3 assistant.py

alloy-voice-assistant's Issues

hey i am new to this

Hello,
I am new to coding, have some patience with me.

i followed your steps and get this error, can somebody help me, i changed nothing on your Assistant.py file
i have a .env file with an open.ai api.
i installed all req. python libs

This is my output in widows:

Traceback (most recent call last):
File "C:\assistant\assistant.py", line 8, in
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
File "D:\Python\Lib\site-packages\langchain\prompts_init_.py", line 32, in
from langchain_core.example_selectors import (
File "D:\Python\Lib\site-packages\langchain_core\example_selectors_init_.py", line 6, in
from langchain_core.example_selectors.length_based import (
File "D:\Python\Lib\site-packages\langchain_core\example_selectors\length_based.py", line 6, in
from langchain_core.prompts.prompt import PromptTemplate
File "D:\Python\Lib\site-packages\langchain_core\prompts_init_.py", line 27, in
from langchain_core.prompts.base import (
File "D:\Python\Lib\site-packages\langchain_core\prompts\base.py", line 22, in
from langchain_core.output_parsers.base import BaseOutputParser
File "D:\Python\Lib\site-packages\langchain_core\output_parsers_init_.py", line 15, in
from langchain_core.output_parsers.base import (
File "D:\Python\Lib\site-packages\langchain_core\output_parsers\base.py", line 18, in
from langchain_core.language_models import LanguageModelOutput
File "D:\Python\Lib\site-packages\langchain_core\language_models_init_.py", line 25, in
from langchain_core.language_models.base import (
File "D:\Python\Lib\site-packages\langchain_core\language_models\base.py", line 75, in
class BaseLanguageModel(
File "D:\Python\Lib\site-packages\pydantic\v1\main.py", line 286, in new
cls.try_update_forward_refs()
File "D:\Python\Lib\site-packages\pydantic\v1\main.py", line 807, in try_update_forward_refs
update_model_forward_refs(cls, cls.fields.values(), cls.config.json_encoders, localns, (NameError,))
File "D:\Python\Lib\site-packages\pydantic\v1\typing.py", line 554, in update_model_forward_refs
update_field_forward_refs(f, globalns=globalns, localns=localns)
File "D:\Python\Lib\site-packages\pydantic\v1\typing.py", line 520, in update_field_forward_refs
field.type_ = evaluate_forwardref(field.type_, globalns, localns or None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Python\Lib\site-packages\pydantic\v1\typing.py", line 66, in evaluate_forwardref
return cast(Any, type_)._evaluate(globalns, localns, set())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'

so, yeah, i would be glad if you can provide help

Parent run d03ad2dd-c64a-499e-bac3-7f6c26544fd9 not found for run 17f371a8-a7d7-4eba-8a41-6c5c1f29b945. Treating as a root run.

Prompt: Hello, can you see me?
Parent run d03ad2dd-c64a-499e-bac3-7f6c26544fd9 not found for run 17f371a8-a7d7-4eba-8a41-6c5c1f29b945. Treating as a root run.

Audio does not work.

Hi ,

First of great project and thanks for making this public. I was trying to run it locally. Upon successful installation of the python packages, the CAM opened up. But then there is an error:

Exception in thread Thread-2 (threaded_listen):
Traceback (most recent call last):
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 1344, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1319, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1365, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1314, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1074, in _send_output
self.send(msg)
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1018, in send
self.connect()
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/http/client.py", line 1460, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/ssl.py", line 1046, in _create
self.do_handshake()
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/ssl.py", line 1317, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
self.run()
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/threading.py", line 989, in run
self._target(*self._args, **self._kwargs)
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/site-packages/speech_recognition/init.py", line 568, in threaded_listen
if running[0]: callback(self, audio)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/Documents/work/ai_projects/video_assistant/assistant.py", line 150, in audio_callback
prompt = recognizer.recognize_whisper(audio, model="base", language="english")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/site-packages/speech_recognition/init.py", line 1412, in recognize_whisper
self.whisper_model[model] = whisper.load_model(model, **load_options or {})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/site-packages/whisper/init.py", line 133, in load_model
checkpoint_file = _download(_MODELS[name], download_root, in_memory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/site-packages/whisper/init.py", line 69, in _download
with urllib.request.urlopen(url) as source, open(download_target, "wb") as output:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 215, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 515, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 532, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 1392, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jaydeep.chakrabarty/anaconda3/envs/v_conda_cam/lib/python3.12/urllib/request.py", line 1347, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)>

Any idea?

AttributeError: 'str' object has no attribute 'model_dump'

Can you tell me the version of the related packages? It seems like a version incompatibility

Video is black on Mac - solved

Machine Apple M2 Max
OS Sonoma Version 14.5

Amazing work - I was very pleasantly surprised when I got it working. I did struggle a little with the fact that when I started the app the video was black. A bit of googling later found the problem to be

def __init__(self):
        self.stream = VideoCapture(index=1)
        _, self.frame = self.stream.read()
        self.running = False
        self.lock = Lock()

When I changed the index from 0 to 1 it just worked. Do you want a PR on the README or are you happy to do it.

My test today was to get it to help me make a cup of tea. the future is not far away - amazing

Your location is not supported by google-generativeai at the moment.

ValueError: Your location is not supported by google-generativeai at the moment. Try to use ChatVertexAI LLM from langchain_google_vertexai.

it's there anyway to use another API ?

Need setup tools for Python 3.12

I needed to install setuptools for this to work with Python 3.12.

python3 -m pip install setuptools