This repo contains a handful of utilities for benchmarking the response latency of popular AI services, including:
Large Language Models (LLMs):
- OpenAI (chat and embedding models)
- Anthropic (chat models)
Text-to-Speech Models (TTS):
- ElevenLabs
- PlayHT
pip install -r requirements.txt
To run a benchmark, first set the appropriate environment variable (e.g., OPENAI_API_KEY, ELEVEN_API_KEY) etc, and then run the appropriate benchmark script, e.g.,
python llm_benchmark.py -m gpt-3.5-turbo "Write me a haiku."
or
python elevenlabs_stream_benchmark.py "Haikus I find tricky, With a 5-7-5 count, But I'll give it a go"
First, install mpv via
brew install mpv
Then, just pass the -p argument when generating text, e.g.,
python playht_benchmark.py -p "Well, basically I have intuition."
You can use the -v parameter to select which voice to use for generation.