Awesome Stable-Diffusion

This is a list of software and resources for the Stable Diffusion AI model.

🖊️ marks content that requires sign-up or account creation for a third party service outside GitHub.
❓ marks content with unclear licensing conditions (e.g. lack of a license on Github)
💵 marks Non-Free content: commercial content that may require any kind of payment.

Due to the fast-moving nature of the topic, entries in the list may be removed at an expedited rate until the ecosystem matures.

See Contributing.

TL;DR

The easiest way to get started for most people is to pick one of the available GUIs based on the desired platform and follow it's installation instructions.

Alternatively, most of the more developed forks (such as InvokeAI) come with their own user interfaces.

To get up and running, one usually needs:

A GUI or CLI based installation on a supported platform (usually NVIDIA GPU but support for Apple Silicon, AMD GPUS and even CPU exists in some branches)
The weights (.ckpt files) for the model downloaded from Huggingface
Sufficent RAM/VRAM available on the system.

Official Resources

Stable Diffusion 2.1 Release Notes - Stable Diffusion 2.1 release notes.
https://github.com/Stability-AI/stablediffusion - Official Stability AI Stable Diffusion 2.1 release.
CompVis/Stable Diffusion - The official release of Stable Diffusion including a CLI, an AI-based Safety Classifier, which detects and suppresses sexualized content, and all the necessary files to get running.
stability-AI/stability-sdk - The official SDK used to build python applications integrated with StabilityAI's cloud platform instead of hosting the model locally. Operation requires an API Key (🖊️💵).
Public Release Announcement - StabilityAI's announcement about the public release of Stable Diffusion.
🖊️ Official Discord - The official Stable Diffusion Discord by StabilityAI.
laion-aesthetic - The dataset used train stable diffusion, useful for querying to see if a concept is represented.

Actively Maintained Forks and Containers

All forks listed here add additional features and optimisations and are generally faster than the original release, as they keep the model in memory rather than reloading it after every prompt. Most forks seem to remove the Safety Classifier which may present a risk if used to provide public-facing services, such as Discord bots. The two most popular forks at this point are Automatic1111 and InvokeAI

AbdBarho/stable-diffusion-webui-docker - Easy Docker setup for SD with multiple user-friendly UI options including AUTOMATIC1111/stable-diffusion-webui, sd-webui/stable-diffusion-webui and invoke-ai/InvokeAI.
AUTOMATIC1111/stable-diffusion-webui - Likely the fastest moving, most feature rich branch at the moment. Gradio based UI with extensive features such as in and outpainting, previews, xy plots, upscaling, clip-interrogation, textual inversion, negative prompting, a variety of upscaling features, training, checkpoint merging and switching capabilities and more. Comes with a handy install script that takes care of most dependencies and addons.
basujindal/stable-diffusion - "Optimized Stable Diffusion"—a fork with dramatically reduced VRAM requirements through model splitting, enabling Stable Diffusion on lower-end graphics cards; includes a GradIO web interface and support for weighted prompts.
bes-dev/stable_diffusion.openvino - A fork for running the model using a CPU compatible with OpenVINO.
DreamArtist - With just one training image DreamArtist learns the content and style in it, generating diverse high-quality images with high controllability. Embeddings of DreamArtist can be easily combined with additional descriptions, as well as two learned embeddings. (standalone version)
fast-stable-diffusion - Speed focused fork with Dreambooth integration.
Gauss - Native MacOS Stable Diffusion App.
imaginAIry - Pythonic generation of stable diffusion images. Unique in that it supports complex text-based masking. Has an interactive CLI, upscaling, face enhancement, tiling, and other standard features. No GUI.
invoke-ai/InvokeAI - (formerly known as lstein/stable-diffusion) - Very active fork adding a conversational CLI, basic web interface and support for GFPGAN, ESRGAN, Codeformer, weighted prompts, prompt blending, negative prompting, img2img, tiling, textual-inversion as well as inference on Apple M1.
KerasCV StableDiffusion - High performance implementation of stable diffusion on KerasCV.
NickLucche/stable-diffusion-nvidia-docker - Multi (Nvidia) GPU capable docker setup of SD
replicate/copg-stable-diffusion - Cog machine learning container of SD v1.4.
stable-diffusion-jupyterlab-docker - A Docker setup ready to go with Jupyter notebooks for Stable Diffusion.
runwayml/stable-diffusion - Stable Diffusion Branch by RunwayML with specifically trained inpainting model for high quality inpainting.

Potentially Stale/Less active branches

These branches have not been updated in more than a months and are kept here for historic purposes

neonsecret/stable-diffusion - Fork focusing on bigger resolutions with less vram at the expense of speed, automatically adjusting to the GPUs abilities. Also includes upscaling, facial restoration via CodeFormer and custom UI

Checkpoints and Weights

Checkpoints (.ckpt files) must be separately downloaded and are required to run Stable Diffusion. The latest stability ai release is 2.1. The last model containing NSFW concepts was 1.5 released by RunwayML.

For 2.x releases, there is a 768x768px resolution capable model trained off the base model (512x512 pixels). You need one of these models to use stable diffusion and generally want to chose the latest one that fits your needs.

🖊️ sd-2.1 Model Cards (768x768px) - Model Cards/Weights for Stable Diffusion 2.1 (diffusion, upscaling and inpainting checkpoints)
🖊️ sd-2.1 Base Model Cards (512px512px) - Model Cards/Weights for Stable Diffusion 2.1 (diffusion, upscaling and inpainting checkpoints) https://huggingface.co/stabilityai/stable-diffusion-2-1-base
🖊️ sd-2.0 Model Cards - Model Cards/Weights for Stable Diffusion 2.0 (diffusion, upscaling and inpainting checkpoints)
🖊️ sd-v1-5 from RunwayML - Stable Diffusion 1.5 Checkpoint released by runwayML.
🖊️ Official Model Card 1.5 - Official Stability AI Model Card on Hugging Face with all versions of the model. Download requires sign-in and acceptance of terms of service.
- stable-diffusion-v-1-4-original.chkpt - The original 1.4 model's card
RealESRGAN Models - Download location for the latest RealESRGAN models required to use the upscaling features implemented by many forks. Different models exist for realistic and anime content. Please refer to the fork documentation to identify the ones you
sd-v1-5-inpainting from RunwayML - Checkpoint optimized for inpainting on SD 1.5, released by runwayML.

Online Demos and Notebooks

HuggingFace/StabilityAI - The official demo on HuggingFace Spaces.
🖊️💵 Offical Colab - The official, optimized colab for running SD on Google Cloud. Due to VRAM requirements required Colab Pro to create images.
andreasjansson/stable-diffusion-animation - Animate between prompts.
Deforum - Advanced notebook for Stable Diffusion with 2D, 3D, Video Input, and Interpolation animations. Includes inpainting, prompt batching, and more.
🖊️ Stable Diffusion Interpolation - AA simple implementation of generating N interpolated images (Colab)
huggingface/diffuse-the-rest - Diffuse the Rest - img2img from simple sketches or uploaded images.

Complementary Models and Tools

Tools and models for use in conjuction with Stable Diffusion

Civitai Model Repository - Repository for custom models and textual inversion embeddings.
Prompt to Prompt - Unofficial Implementation of Cross-attention-control for prompt to prompt image editing.
sd-prompt-graph - This is a React-based curve editor GUI for prompt interpolation animations made with Stable Diffusion.
DAAM - Diffusion attention attribution maps, generating heatmaps modelling the impact of specific terms and tokens in the prompt on the final diffusion result.
sd-concepts-library - A library of user created textual-inversion embeddings to add new concepts to stable diffusion

Training

Dreambooth-Stable-Diffusion - Implementation of Google's DreamBooth for stable diffusion, allowing fine-tuning of the model for specific concepts.
LORA - LORA (Low-rank Adaptation) for Faster Text-to-Image Diffusion Fine-tuning (UNET+CLIP), a faster, more efficient Fine tuning method requiring less RAM than Dreambooth. Huggingface demo here.
Stable Tuner - Stable Tuner, an easy to install Dreambooth trainer with a very comfortable user interface.
Stable Diffusion Trainer - Stable Diffusion trainer with scalable dataset size and hardware usage. Requires 10G of VRAM.
textual-inversion - Addition of personalized content to Stable Diffusion without retraining the model (Paper, Paper2).
Stable Dreamfusion - Text to 3D dreamfusion implementation based on stable diffusion.

GUIS

Most of these GUIS, unless mentioned otherwise in their documentation, include stable-diffusion.

🖊️💵 Auto SD Workflow - A UI for lstein/stable-diffusion's dream.py with optimized UX for large-scale/production workflow around image synthesis. Video Walkthrough.
Carefree Creator (local version) - User friendly GUI with a creator/artist centric workflow.
cmdr2/stable-diffusion-ui - Another, simple to use UI for windows and Linux.
DiffusionBee - Self contained binary app for MacOS.
🖊️ DiffusionUI - web UI made with Vue.js inspired by Dall-e using diffusers, perfect for inpainting. Video demo
🖊️ KIRI.ART (formerly SD-MUI) - mobile-first PWA with multiple models (incl. waifu diffusion). Run free locally or use free & paid credits on the live site. Built with React + MaterialUI. (Source Code) MIT License TypeScript
💵 NMKD GUI - Windows UI, fully featured. Closed source. Pick your own price.
sd-webui/stable-diffusion-webui - Very active fork with optional, highly featureful Gradio UI and support for txt2img, img2img inpainting, GFPGAN, ESRGAN, weighted prompts, optimized low memory version, optional textual-inversion and more.
Stable Diffusion GRisk GUI - Windows GUI binary for SD. Closed source so use at your own risk.
Stable Diffusion Infinity - A proof of concept for outpainting with an infinite canvas interface. (requires powerful GPU).
Unstable Fusion - A Stable Diffusion desktop frontend with inpainting, img2img and more
stable-diffusion-webui-docker - A docker based frontend integrating the most popular forks.
🖊️ Synesthetic - Prompt search engine and generator focused on remixing

Upscaling

BasicSR - Open-source upscaling and restoration toolbox supporting several models.
BSRGAN - BSRGAN—another upscaling solution specialized in upscaling degraded images.
Cupscale - GUI for running various upscaling models, including ESRGAN and RealESRGAN.
RealESRGAN - ESRGAN Upscaling (2x, 4x) and content restoration. Python: pip install realesrgan
jquesnelle/txt2imghd - A port of the GOBIG mode from progrockdiffusion, providing high quality upscaling on top of txt2img.
Upscale Wiki Model Database - Wiki and model database for checkpoints compatible with ESRGAN et al.

Content Restoration

lama-cleaner - Content aware AI inpainting tool useful for removing unwanted objects or defects from images. Python: pip install lama-cleaner
GFPGAN - Face Restoration GAN included in several forks for automatically fixing the face deformation commonly found in SD output.
CodeFormer - Another Face Restoration model (Paper).

Task Chaining

chaiNNer - Graphical node-based editor for chaining image processing tasks.
ai-art-generator - AI art generation suite combining Stable Diffusion and other models for high volume art generation.
dfserver distributed backend AI pipeline server for building self-hosted distributed GPU cluster to run the Stable Diffusion and various AI image or prompt building model.

Prompt Building

Prompts are the instructions given to diffusion models to manipulate their output.

Stable diffusion prompt book - OpenAI's stable diffusion prompt book, a very comprehensive resource on prompt engineering.
ai-art.com/modifiers - A visual reference guide for keywords.
aipromptguide.com - Visual Database of styles, modifier, artists and persons
🖊️ aiTransformer Prompt Builder - Help build text prompts intuitively using an Image To Text tool to generate detailed text descriptions from image, and 800+ special Textual Inversion terms and often-used generic terms with sample images.
krea.ai - Prompt search engine that also recommends similar prompts to the one that you click on.
- 🖊️ With account creation, you can like and save prompts in your own collections.
Lexica.art - A searchable, visual database of images and the prompts settings used to create them.
pharmapsychotic/clip-interrogator - Jupyter notebook uses CLIP models to suggest a prompt for images similar to a given image (Demo).
🖊️Phraser - A visual prompt builder drawing on a database of examples. (Requires account creation)
🖊️Prompthero = Another visual prompt builder and reference library.
PromptoMania - A visual prompt construction tool.
rom1504/clip-retrieval - Searches for prompt keywords in the datasets used in training Stable Diffusion and other models (Online GUI). Some GUIS like Automatic1111 include this functionality.
Stable Diffusion Prompt Generator - Gives suggestions for improving a given text prompt.
PromptSearch - Yet another Stable Diffusion search engine but with public API
Same Energy - A visual search engine that returns images that have the same 'energy'.
PublicPrompts - Collection of PublicPrompts

Specialized Usecases

dream-textures - A blender addon leveraging stable diffusion for texture creation.
stable-dreamfusion - An open source implementation of Google's text-to-3D dreamfusion paper with imagegen replaced by stable diffusion.

Tutorials and Comparisons

Tutorials and high quality educational resources

Getting Up and Running

Stable Diffusion How To - A basic tutorial on getting Stable Diffusion up and running.
Installing on Windows - A guide on installing and runing Stable Diffusion on Windows.
Running on M1 Apple Silicon - Reddit thread with instructions on running Stable Diffusion on Apple M1 CPU and GPU.
Easy CPU-only Stable Diffusion - A guide on setting up CPU-only Stable Diffusion for GNU/Linux without littering the system with dependencies.
"Ultimate GUI Retard Guide" - Tutorial for installing the sd-webui fork.

Learning and Mastering

Stable Diffusion Akashic Records - A comprehensive curated list of guides, studies, keywords, prompts and artists.
Sunny's Tips & Tricks - Sunny's SD Tips & Tricks Google Doc with lots of visual comparisons and useful information.
AI Image Generator Comparison - A visual comparison between Dall-e, Stable Diffusion and Midjourney by PetaPixel.com.
Getting great results at Stable Diffusion - A guide on generating images that don't suck.
Illustrated Stable Diffusion - An illustrated primer on how Stable Diffusion works.
Practical deep learning for coders - high quality course by fast.ai aimed at coders that covers many aspects of deep learning, including stable-diffusion.
Top SD Artists - Searchable list of artists known by Stable Diffusion with example images.

Studies

Modifier Studies - Visual study of popular modifiers/keywords.
Artist Studies - Visual study of various artists.

Extending Functionality

Building a SD Discord Bot - A tutorial on building a Stable Diffusion Discord bot using Python.

Community Resources

1 week of Stable Diffusion - A curated list of Stable Diffusion services, adaptations, user interfaces and integrations.
pharmapsychotic.com/tools - A curated list of Tools and Resources for AI Art, including but not limited to Stable Diffusion.
Stable Diffusion Resources - A thorough resource for answering pressing questions about Stable Diffusion, including guides, tutorials, and best software.

Social Media

r/StableDiffusion - Stable Diffusion Subreddit. (Semi-official)
r/sdforall - SDForAll
Diffusion Pulse - Weekly Stable Diffusion newsletter

Plugins for third party apps

Blender Plugin - Plugin for the free 3D modelling software Blender
Gimp Plugin - Gimp Plugin.
Krita Plugin - A krita and Gimp SD plugin
Krita 5.0 Plugin - Another Krita Plugin based on the popular Automatic1111 fork.
🖊️ Photoshop Plugin - SD for Photoshop (Adobe Exchange)

Commercial SaaS and apps implementing Stable Diffusion

🖊️💵 AI Art Generator (IOS) - iOS App to generate art using Stable Diffusion.
aiimagegenerator.org - Free online art generator for generating images using Stable Diffusion. Supports txt2img and img2img generation and inpainting.
🖊️ aiTransformer - Free online image/video transformation tools, including Super Stylizer based on Stable Diffusion, supports txt2img, img2img and img2txt (clip-interrogator).
🖊️💵 Barium.ai - Generate PBR (physics based rendering) textures from text. Free and paid plans.
🖊️💵 Canva text-to-image - Text-to-image (beta) service from Canva
🖊️💵 Dream Studio - Online art generation service by StabilityAI, the creators of Stable Diffusion. Similar to services like DALL-E or Midjourney, this operates on a credit model with a free allowance of credits given to signed up users on a monthly basis.
🖊️💵 dream.ai - Online art generation service by Wombo.ai (mobile apps available).
🖊️💵 GimmeAPIs - Third party REST API Stable Diffusion service.
🖊️💵 Image Computer - Easy-to-use service aimed at non-technical people (comes with free trial credits)
🖊️💵 Neural.love - Another online art generator with generous free credits as of Oct 2022.
🖊️💵 replicate.com stable diffusion - Another SaaS offering for Stable Diffusion.
🖊️💵 Starry AI (IOS) - Another IOS app offering stable diffusion with preset art styles.
🖊️ Stable Horde - Distributed stable diffusion cluster (think folding@home) with web, discord and telegram interfaces where joining with your GPU gives you priority.
🖊️💵 Stable Diffusion as API - Third party REST API into table Diffusion service.

The Attic - previous links, retired for inactivity

lowfuel/progrock-stable - Fork with optional Web GUI and a different approach to upscaling (GoBIG/ESRGAN)
- txt2imghd - Fork of progrock diffusion that creates detailed, higher-resolution images by first generating an image from a prompt, upscaling it, then running img2img on smaller pieces of the upscaled image, and blending the results back into the original image.

awesome-stable-diffusion / awesome-stable-diffusion Goto Github PK