karafra / ai-art

Discord bot generating images from given query

AI Art

Discord bot generating AI art collages
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Deployment
4. Usage
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

Simple discord bot which generates collages based on any query you give it (most of the time). This bot uses model which you can find here

(back to top)

Built With

• Typescript
• Discord.js
• Collage
• Canvas
• amqp-client.js
• Sentry.io
• Jest
• Codecov
• Docker
• NestJs
• Compodoc
• Mongo
• TypeORM
• GraphQl

(back to top)

Getting Started

Prerequisites

This is an example of how to list things you need to use the software and how to install them.

• npm

  npm install npm@latest -g

Installation

1. Create Discord application on a free API Key at https://discord.com/developers/

2. Click on Bot tab and save your token.

3. Clone the repo

   git clone https://github.com/karafra/ai-art.git

4. Install NPM packages

   npm install

5. Enter your API key and bot id into config.yml.

   sentry:
     dsn: "{SENTRY_DSN}"
   # This one is optional ... only if you want to use ai-story command
   openAi:
     token: ...
   amqp:
     url: "{AMQP_URL}"
   discord:
       token: "{DISCORD_TOKEN}"
   deploy:
     port: "{PORT}"

   Configuration file supports simple environment variable substitution in format "{VARIABLE_NAME}", where parentheses are required.

(back to top)

Deployment

Method 1: Deployment to Heroku

Recommended method of deploying this bot is deployment on Heroku. To deploy to Heroku please click on button bellow.

IMPORTANT

After successful deployment you will have to switch dyno from web to worker. If you do not do this, app will not bind to port and fail.

After successful deployment you can invite bot to your server by clicking on this link https://discord.com/api/oauth2/authorize?client_id={CLIENT_ID}&permissions=34816&scope=applications.commands%20bot, where CLIENT_ID is your bots client id. Link already contains minimal scopes (bot, application.commands) and minimal bot permissions (send messages, attach files)

Method 2: Containerized deployment:

Another even easier method of deployment is deployment via docker container.

1. Verify docker-compose installation

   A] Type docker-compose -v into terminal. if output looks similar to docker-compose version 1.29.2, build 5becea4c then you can continue to the next step.

   B] If this command throws an error, you have to follow docker-compose installation guide

2. Set required variables

   • Only required variables is TOKEN, this can be set as environment variables using export ENV_NAME=VALUE on linux based OS or $env:VARIABLE_NAME=VALUE on Windows based OS

3. Building Docker containers

• Type docker-compose -f "docker/deploy/docker-compose.yml" build into terminal, this will automatically build all required docker images.

4. Start container

   • Type docker-compose -f "docker/deploy/docker-compose.yml" up into terminal. This will start all services needed. RabbitMQ management console will be accessible here with login credentials being:

     • username:
       • guest
     • password:
       • guest

   • MongoDb management console will be accessible here without any login credentials

This network is not external, so it will not be accessible from outside.

(back to top)

Usage

Commands are separated into 2 command groups

• help - Help command

• /ai-art

  • cog-view-2 - Generates collage of 9 images using CogView2 model
  • dalle-mini - Generates collage of 9 images using Dall-e mini model
  • wombo-dream - Generartes one image based on WomboDream model

• /ai-story

  • story - Generates story from given headline (Requires OpenAi API) token

• React with ✉️ to any collage and bot will dm it to you.

Discord offers autocompletion so all you need is to start typing name of the command or group in which command is, discord will then guide you through all the required parameters using its autocompletion.

For more examples, please refer to the Documentation

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the Apache2.0 License. See LICENSE for more information.

(back to top)

Contact

Your Name - @Karafro - [email protected]

Project Link: https://github.com/karafra/ai-art

(back to top)

Acknowledgments

• DALL·E Mini
  • Image generation model for /ai-art dalle-mini
• CogView2
  • Image generation model for /ai-art cog-view-2
• Open Ai
  • Story generation model for /ai-story story
• WomboDream
  • Art generation model for ai-art wombo-art

(back to top)

ai-art's Issues

Fix CogView2 command

TypeError: Cannot destructure property 'dtoInstance' of 'commandNode' as it is undefined.

I successfully deployed the app using docker-compose -f "docker/deploy/docker-compose.yml" up -d.

But when I interact with the bot with /ai-story story, it crashes and I see the following error in the logs.

2023-01-24 16:05:09 This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
2023-01-24 16:05:09 TypeError: Cannot destructure property 'dtoInstance' of 'commandNode' as it is undefined.
2023-01-24 16:05:09     at CommandExplorer.<anonymous> (/usr/src/app/node_modules/@discord-nestjs/core/src/explorers/command/command.explorer.ts:90:17)
2023-01-24 16:05:09     at Generator.next (<anonymous>)
2023-01-24 16:05:09     at /usr/src/app/node_modules/@discord-nestjs/core/dist/explorers/command/command.explorer.js:17:71
2023-01-24 16:05:09     at new Promise (<anonymous>)
2023-01-24 16:05:09     at __awaiter (/usr/src/app/node_modules/@discord-nestjs/core/dist/explorers/command/command.explorer.js:13:12)
2023-01-24 16:05:09     at Client.<anonymous> (/usr/src/app/node_modules/@discord-nestjs/core/src/explorers/command/command.explorer.ts:67:76)
2023-01-24 16:05:09     at Client.emit (node:events:525:35)
2023-01-24 16:05:09     at Client.emit (node:domain:489:12)
2023-01-24 16:05:09     at InteractionCreateAction.handle (/usr/src/app/node_modules/discord.js/src/client/actions/InteractionCreate.js:83:12)
2023-01-24 16:05:09     at Object.module.exports [as INTERACTION_CREATE] (/usr/src/app/node_modules/discord.js/src/client/websocket/handlers/INTERACTION_CREATE.js:4:36)
2023-01-24 16:05:09     at WebSocketManager.handlePacket (/usr/src/app/node_modules/discord.js/src/client/websocket/WebSocketManager.js:346:31)
2023-01-24 16:05:09     at WebSocketShard.onPacket (/usr/src/app/node_modules/discord.js/src/client/websocket/WebSocketShard.js:478:22)
2023-01-24 16:05:09     at WebSocketShard.onMessage (/usr/src/app/node_modules/discord.js/src/client/websocket/WebSocketShard.js:317:10)
2023-01-24 16:05:09     at WebSocket.onMessage (/usr/src/app/node_modules/ws/lib/event-target.js:199:18)
2023-01-24 16:05:09     at WebSocket.emit (node:events:513:28)
2023-01-24 16:05:09     at WebSocket.emit (node:domain:489:12)

error

I can't figure this one out.

DALLE 2 or other art ai support

Could you possibly add either DALLE 2, synth.run, or CLIP?

Add web client

MongoDB issue

I just redeployed the bot with the latest commits and got this error immediately after. How do I remediate this?

2022-08-26T05:19:52.089809+00:00 app[worker.1]: MongoError: Collection already exists. NS: test.Job 2022-08-26T05:19:52.089810+00:00 app[worker.1]: at MessageStream.messageHandler (/app/node_modules/mongodb/lib/cmap/connection.js:299:20) 2022-08-26T05:19:52.089810+00:00 app[worker.1]: at MessageStream.emit (node:events:513:28) 2022-08-26T05:19:52.089810+00:00 app[worker.1]: at MessageStream.emit (node:domain:489:12) 2022-08-26T05:19:52.089811+00:00 app[worker.1]: at processIncomingData (/app/node_modules/mongodb/lib/cmap/message_stream.js:144:12) 2022-08-26T05:19:52.089811+00:00 app[worker.1]: at MessageStream._write (/app/node_modules/mongodb/lib/cmap/message_stream.js:42:5) 2022-08-26T05:19:52.089811+00:00 app[worker.1]: at writeOrBuffer (node:internal/streams/writable:391:12) 2022-08-26T05:19:52.089811+00:00 app[worker.1]: at _write (node:internal/streams/writable:332:10) 2022-08-26T05:19:52.089812+00:00 app[worker.1]: at MessageStream.Writable.write (node:internal/streams/writable:336:10) 2022-08-26T05:19:52.089812+00:00 app[worker.1]: at TLSSocket.ondata (node:internal/streams/readable:754:22) 2022-08-26T05:19:52.089812+00:00 app[worker.1]: at TLSSocket.emit (node:events:513:28) 2022-08-26T05:19:52.089813+00:00 app[worker.1]: at TLSSocket.emit (node:domain:489:12) 2022-08-26T05:19:52.089813+00:00 app[worker.1]: at addChunk (node:internal/streams/readable:315:12) 2022-08-26T05:19:52.089813+00:00 app[worker.1]: at readableAddChunk (node:internal/streams/readable:289:9) 2022-08-26T05:19:52.089813+00:00 app[worker.1]: at TLSSocket.Readable.push (node:internal/streams/readable:228:10) 2022-08-26T05:19:52.089814+00:00 app[worker.1]: at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) 2022-08-26T05:19:55.404560+00:00 app[worker.1]: �[32m[Nest] 22 - �[39m08/26/2022, 5:19:55 AM �[32m LOG�[39m �[38;5;3m[createCappedJobsCollection1661353047612] �[39m�[32mRunning migration Create capped Jobs collection 1661353047612�[39m 2022-08-26T05:19:55.404785+00:00 app[worker.1]: �[95m[Nest] 22 - �[39m08/26/2022, 5:19:55 AM �[95m DEBUG�[39m �[38;5;3m[createCappedJobsCollection1661353047612] �[39m�[95mCreating capped collection "Jobs" with size 400000000B�[39m 2022-08-26T05:19:55.440021+00:00 app[worker.1]: Migration "Create capped Jobs collection 1661353047612" failed, error: Collection already exists. NS: test.Job 2022-08-26T05:19:55.441449+00:00 app[worker.1]: �[31m[Nest] 22 - �[39m08/26/2022, 5:19:55 AM �[31m ERROR�[39m �[38;5;3m[TypeOrmModule] �[39m�[31mUnable to connect to the database. Retrying (2)...�[39m

Containerized deployment ERROR [TypeOrmModule] Unable to connect to the database.

deploy-discord-chat-bot-1  | [Nest] 31  - 01/18/2023, 3:42:14 PM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (3)...
deploy-discord-chat-bot-1  | MongoServerSelectionError: connect ECONNREFUSED 127.0.0.1:27017

on step
docker-compose -f "docker/deploy/docker-compose.yml" up

any ideas?

🗒️ Add documentation for commands

Add command documentation

• Create tsconfig.doc.json file in the repo root

  • Move all configuration from package.jsom build:docs command to this file

• Add support for commands section in documentation

  • For how to do this please refer to this article

• Create markdown file with static command documentation (possible update to dynamic annotation based generator later)

Add Docker images for models

Persisted queries are enabled and are using an unbounded cache. Your server is vulnerable to denial of service attacks via memory exhaustion. Set `cache: "bounded"` or `persistedQueries: false` in your ApolloServer constructor, or see https://go.apollo.dev/s/cache-backends for other alternatives.

Add guide for other config vars

I'm mostly concerning these two that are required

Stable diffusion API

I know it's paid (like openAI story anyway) but it would be nice to integrate the stable diffusion API
https://github.com/Stability-AI/stability-sdk

Apparently, you can run this model locally for free too
https://github.com/CompVis/stable-diffusion

https://www.howtogeek.com/830179/how-to-run-stable-diffusion-on-your-pc-to-generate-ai-images/

Command "/ai-story story" is missing and "/ai-story open-ai" doesn't work.

I successfully deployed the app using docker-compose -f "docker/deploy/docker-compose.yml" up -d.

In my .env, I have added OPEN_API_TOKEN = '[redacted]' which contains a valid API key from Open AI.

However, the command to run /ai-story story doesn't show up.

If I use /ai-story open-ai Give me a poem instead, I get the following error in my logs.

2023-01-24 16:52:06 [Nest] 32  - 01/24/2023, 5:52:06 AM   DEBUG [OpenAiCommand] /ai-story open-ai command called
2023-01-24 16:52:07 [Nest] 32  - 01/24/2023, 5:52:07 AM   DEBUG [AiStoryService] Generating Give me a poem in style text-davinci-002
2023-01-24 16:52:07 [Nest] 32  - 01/24/2023, 5:52:07 AM   ERROR [OpenAiCommand] Open-ai command failed with exception: Could not generate art "Give me a poem" based on AiStoryService in style "text-davinci-002"

The only command that reliably works is /ai-art wombo-dream.

AI-Story Failing Error 401

Have the bot deployed via Docker.
OpenAI key is valid and functioning tested by sending a request to https://api.openai.com/v1/models with my API key in the header and it did return the list of available models so that seems to be working fine.

I did slap the tokens into the docker-compose file but that hasn't been an issue for me for the required bot tokens.
Tried with and without quotation marks for the OpenAI token.

  args:  
   - DISCORD_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  
   - DISCORD_BOTID=111111111111111111  
   - OPEN_AI_TOKEN=xx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

When sending the story request I get the following error in Discord:
Could not process prompt due to internal error (Request failed with status code 401)

Of course, the Docker log also reports:
2022-07-14 06:45:05 : Generating story with headline a storm is brewing based on text-davinci-002 model
2022-07-14 06:45:06 : Request failed with status code 401

Discord bot is down

it's been down for quite a bit and i wonder if it will ever come back