olaviinha / neuraltexttoimage

Colabs for text prompt steered image generators

Colab notebooks for text-to-image generators

User-friendly Colab notebooks for various text prompt steered synthetic image generators. These notebooks are not actively maintained, though issues and pull requests are welcome.

Available notebooks (latest to oldest):

• Stable Diffusion (v1.4, v1.5, waifu diffusion)
• DALL·E Mini (craiyon.ai)
• Latent Diffusion LAION 400M
• ruDALLE Malevich
• JAX CLIP Guided Diffusion
• CLIP+VQGAN
• Latent Vision
• Big Sleep

---

These techniques are all more or less descendants of Big Sleep, the original concept of turning text into imagery by using CLIP (OpenAI) to steer BigGAN (Andrew Brock et al.), as introduced by Ryan Murdock in his original notebook. The technique has since branched and evolved.

There are countless versions of these notebooks out there. The point of this set of notebooks is a user-friendly and quick-to-run UI. Some additional basic features have also been added to some of the notebooks, such as saving all/selected progression images, saving progression video, batch processing, etc.

I do not use these notebooks on a regular basis, and hence I'm mostly unaware when some library gets updated and a notebook may no longer function; Please do raise an issue should any problems occur.

---

Stable Diffusion by CompVis, Stability AI and LAION

This colab requires registering with Hugging Face for login token and accepting model card terms to download required models here and here.

"Satanism in the 1900s"

---

DALL·E Mini (craiyon.ai) by Boris Dayma

This colab requires registering with wandb for an authentication key to download required models.

"Satanism in the 1900s"

---

Latent Diffusion LAION 400M by CompVis

"Satanism in the 1900s" (upscaled)

---

ruDALLE Malevich by Sberbank-AI

"Satanism in the 1900s"

---

JAX CLIP Guided Diffusion by nshepperd

Method originally by Katherine Crowson

300 iterations of "Satanism in the 1900s".

---

CLIP+VQGAN by Katherine Crowson

Papers: 1) VQGAN by Patrick Esser et al, 2) CLIP by OpenAI

400 iterations of "Satanism in the 1900s".

---

Latent Vision by Ryan Murdock

Papers: 1) VQGAN by Patrick Esser et al, 2) CLIP by OpenAI

400 iterations of "Satanism in the 1900s".

---

Big Sleep by Phil Wang (more precisely this fork)

Papers: 1) BigGAN by Andrew Brock et al, 2) CLIP by OpenAI

Original Big Sleep method.

400 iterations of "Satanism in the 1900s".

neuraltexttoimage's Issues

Latent Diffusion LAION 400M colab not working

Latent diffusion colab does not seem to be working:

Loading model from /content/drive/MyDrive//image-generation//latent_diffusion_txt2img_f8_large.ckpt
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-2-fec7cfd0875d>](https://localhost:8080/#) in <module>()
    168 
    169 config = OmegaConf.load("configs/latent-diffusion/txt2img-1p4B-eval.yaml")
--> 170 model = load_model_from_config(config, f"{model_path}/latent_diffusion_txt2img_f8_large.ckpt")
    171

I've given access to the drive, is there any other requirement?

Thank you for the support!

LatentVision broken

Reported by user:

FileNotFoundError: The directory '/content/fauxdrive/output/symtol_UponStrangeAeons/progress_1' does not exist

Reminder to self to check at some point. Looks like something goes wrong if you don't use Drive.