parsehex / buddygenai

Everyone needs a buddy -- make one and chat with them using BuddyGenAI

BuddyGenAI is a chat application designed to create and interact with virtual buddies

✅ Features

• Generate Buddies to chat with 🤖
• Create a detailed Buddy from just a few words 📝
  • Or customize your Buddy to behave exactly how you want 🎨
• Generate profile pictures for your Buddy 🖼️

Support / Help

If you find an issue with the app, please report it on the issues page. If you need help setting up the app or anything else, feel free to ask on the discussions page.

⚙️ Setup

BuddyGenAI is in pre-release stage and so the experience will be unstable at times. If you encounter any issues, please report them on the issues page.

You may check the releases page to get a pre-built version of the app, or follow the section below to build it yourself.

Building

(section is incomplete/outdated -- e.g. project is using llama.cpp again instead of llamafile)

Acquiring Binaries

First you'll need to place binaries in the ./binaries/ folder. You can build them yourself or download them from their releases page. You'll need:

• LlamaFile - ./binaries/llamafile or ./binaries/llamafile.exe
• Stable-Diffusion.cpp - You should at least have a ./binaries/stable-diffusion.cpp/cuda12 folder with the stable-diffusion binary in it. Besides cuda12, the following are also checked at runtime in this order:
  • rocm5.5
  • clblast
  • vulkan
  • avx512
  • avx2
  • avx
  • noavx

Steps to Build BuddyGenAI

Node v18 is recommended. I personally use nvm or NVM for Windows to manage Node versions.

You'll need to download the latest release of LlamaFile. Place the binary at ./binaries/llamafile and make it executable (for Windows: add ".exe" to the end of the file's name).

git clone https://github.com/parsehex/BuddyGenAI

cd BuddyGenAI

# Install dependencies
npm install

# ./scripts/[linux or windows]/setup_llamacpp.[sh or ps1]
./scripts/[linux or windows]/setup_stablediffusioncpp.[sh or ps1]
# ./scripts/[linux or windows]/setup_whispercpp.[sh or ps1]

# Start the app in development mode (in electron)
npm run dev:electron

code .

📡 Usage

Development

Commonly Used Scripts:

# Start the app in development mode (in electron)
npm run dev:electron
# 'npm run dev:electron:win' for windows

# same as dev:electron but deletes db first
npm run reset-dev
# npm run reset-dev:win

# Start the app in development mode (in browser)
npm run dev

Developer Notes

• There are 2 AppSettings.ts files. One in electron/ and one in lib/api/. The electron one is to get settings when running LlamaFile/SDCPP and so it doesn't do any saving of settings. The one in lib/api/ is for the app UI and does save settings.
  • Additionally, stores/main.ts includes a copy of the Settings interface and thus also needs updated when changes are made to AppSettings.ts.
• Apologies for the lack of testing and the overall messiness of the project.
  • Several refactors are needed.
    • Lots of duplicated code (AppSettings, anything else shared between electron/client)
    • Project minimally uses Vercel's AI SDK (really just for message streaming), I want to re-implement to avoid the need for a server (just llama.cpp server then) and reduce dependencies.
    • Accessing data from db is a mess, want to use tRPC with electron's IPC to reduce complexity.
  • Planning to use Vitest for testing before undergoing major refactors.

License

This project is licensed under the MIT License. See the LICENSE file for more details.

Copyright (C) 2024 Thomas Mays

All AI Models are licensed under their respective licenses. See the Licenses folder for more details.

buddygenai's Issues

Next Release Notes

Figure it's a good idea to put down some of the more glaring issues to address for the next release, whenever that is.

• Database updating/migrating - Need to do or users have to lose DB to update ☹️
  • The demo's DB isn't setup for migrating to a new schema there's some code there for it but it never seemed to work).
• Updating
  • idk how updating electron works with/thru itch.io
  • At least linking to the game/other pages to get latest version
  • Display the current version in-app
  • Could possibly check for new version but user needs to manually download
• Fix navigation (e.g. Clicking on consecutive buddies in the sidebar doesn't go to their page unless you Ctrl+R to reload the page)
  • I believe this is related to Vue state issues (e.g. I have some local state where I should be loading from the global state/store, not using Vue's refs everywhere I should)

Refactor Away from Nuxt

This will cut down on the # of dependencies by a lot.

• use file-based router for vue (what is that one, unplugin-vue-router maybe?)
• re-implement color mode

Issues with unfocusing window while buddy is responding

If you send a message and un-focus the window while the buddy is still responding, the app will be in a broken state when you return.

I've worked around the problem by reloading the page if app's in a detected bad state. In that case then the user comes back to same-ish screen and sees 1 button enabled by default -- the reload button, which is fixed to work for the user message being the one reload (and so adds AI msg instead of updating).

A few thoughts come to mind to address this:

• Refactor to handle doing DB operations server side and save the message properly once response is finished.
• Refresh renderer/client-side to get updated message
  • Might/Probably can't resume streaming the response, must wait until it's finished
    • Idea: Client can figure out if message is incomplete yeah? Then fire off an eventsource request (to new endpoint) to get message after loading
      • If message still loading, endpoint would first send back like { wait: true } and eventually the message itself, or just { wait: false } and the message immediately
      • Client can replace the message content with a spinner (or do a spinner overlay overtop original text)

Refactor Build + Update Process

Goals:

• Verify that app works with no binaries present
  • Already know if won't work for Local, but External should work?
• Setup automatic builds with GH Actions
  • How are Actions disabled?
• Setup/Verify updating via GH build
  • Re-enable updater code
• Setup separate updating for Itch.io build
  • I don't think we'll use CI, so just an update server (probably use serverless somehow)