This code is licensed under GPL for noncommercial use. For commercial use / custom game engine integration contact me at crcdng.com
This is a demo for an audiovisual novel system, integrating ElevenLabs text-to-speech synthesis with the Ren'Py visual novel engine. Made in 48 hours during the LabLab ElevenLabs AI hackathon.
A visual novel is a form of interactive fiction, one that combines text, images and animations. Ren'Py is a popular and open source visual novel engine based on Python. It has been used to create thousands of interactive stories, visual novels and games, including Andromeda Six, Highway Blossoms and Doki Doki Literature Club.
I am using the Ren'Py automatic voice feature to demonstrate adding ElevenLabs voices to a visual novel. There are two Python scripts (for now) that do the job: generate_voices.py and wav_to_ogg.py.
You can check out the folder avdemo
for the demo. The web demo is at https://github.com/crcdng/avnovel_demo
You can play it live here: https://crcdng.github.io/avnovel_demo/
To try out the integration for yourself, follow these steps:
- An editor such as VSCode with the Ren'Py extension
- Ren'Py
- A Python 3 environment - I recommend using conda
- The ElevenLabs Python API. Add it to your Python enviroment with
pip install elevenlabs
- ffmpeg to convert audio from
.wav
to.ogg
- Prepare a game / interactive fiction / visual novel in Ren'Py
- Extract the dialogue data via the Ren'Py menu
Extract Dialogue
. This creates a filedialogue.tab
in your project directory. - Open this file in your editor. Note the entries in column "Character". The characters that are listed here, are defined in your script, e.g.
define a = Character("Aoki")
. An empty entry stands for the narrator. Add a "-" in each line that you don't wan't to generate. Also go through theDialogue
column and clean up any markup that might be there. You will send these strings to the ElevenLabs API. - Open
generate_voices.py
. Provide your ElevenLabs API key via the environment variable ELEVEN_API_KEY or via the argumentapi_key
in thegenerate()
function (optional). Edit the character-to-voice mapping in functiongetVoice()
according to step 3. You have to use ElevenLabs voices that are available for your access level. If you don't have an API key, use voices included the free tier. You can check this by runningpython list_voices.py
. Test the voices here: https://elevenlabs.io/speech-synthesis - Configure Ren'Py to use the automatic voice system by adding the line
define config.auto_voice = "voice/{id}.ogg"
in the Ren'Py fileoptions.rpy
- Run
python generate_voices.py [your Ren'Py project directory]
. This will create a subdirectoryvoice
inside of thegame
dirctory, call the Elevenlabs API to generate the audio files and store them with the identifier names fromdialogue.tab
.
- Run
python wav_to_ogg.py [your Ren'Py game directory]/voice
to convert the audio files from.wav
to.ogg
format. You can choose to keep the original files (default) or delete them bz adding the-d
switch.
- In Ren'Py, reload the scripts with
Shift-R
and restart your visual novel. You should now hear the voices automatically added to the lines spoken by the characters and the narrator โ
Note these scripts, demo and accompanying materials were made in 48 hours in the LabLab hackathon while listening to techno (Tommorrowland).
Possible code improvements / Next steps:
- eliminate some of the manual steps above
- add more features, e.g multilingual support
- add a UI
- improve the code quality
- add tests
Characters for the demo project: kid-blue Backgrounds for the demo project: Ren'Py example backgrounds
Ren'Py license: https://www.renpy.org/doc/html/license.html