somewheresy / s2ml-generators Goto Github PK
View Code? Open in Web Editor NEWMultiple notebooks which allow the use of various machine learning methods to generate or modify multimedia content
License: MIT License
Multiple notebooks which allow the use of various machine learning methods to generate or modify multimedia content
License: MIT License
Hi everything is working but facehq model won't run for some reason - error message is -
ModuleNotFoundError Traceback (most recent call last)
in ()
77 print('Using seed:', seed)
78
---> 79 model = load_vqgan_model(args.vqgan_config, args.vqgan_checkpoint).to(device)
80 perceptor = clip.load(args.clip_model, jit=False)[0].eval().requires_grad_(False).to(device)
81
12 frames
/usr/lib/python3.7/importlib/_bootstrap.py in find_and_load_unlocked(name, import)
ModuleNotFoundError: No module named 'taming.modules.misc'
any fix for this?
This class is defined twice. I think one version is intended for CLIP without diffusion? if so perhaps different names for the classes is the solution?
Hello,
It seems the-eye.eu is unreachable, therefore I cannot download the 512x512 unconditional diffusion model, and it is not provided on the openai repo (there is only the 256x256 one)
Any idea where i could find the model ?
Thanks a lot
i am trying to generate a image by collab clound processors, but it keeps disconnecing after 2 to 3 hours, i have tryed to use autoclick, but it dies nor work, some one knows how to stay more time processing?
Currently when adjusting the image size parameter on the guided diffusion model in the s2ML notebook, it will fail with the following error:
RuntimeError: Error(s) in loading state_dict for UNetModel:
Missing key(s) in state_dict: "input_blocks.7.0.skip_connection.weight", "input_blocks.7.0.skip_connection.bias", "input_blocks.7.1.norm.weight", "input_blocks.7.1.norm.bias", "input_blocks.7.1.qkv.weight", "input_blocks.7.1.qkv.bias", "input_blocks.7.1.proj_out.weight", "input_blocks.7.1.proj_out.bias", "input_blocks.8.1.norm.weight", "input_blocks.8.1.norm.bias", "input_blocks.8.1.qkv.weight", "input_blocks.8.1.qkv.bias", "input_blocks.8.1.proj_out.weight", "input_blocks.8.1.proj_out.bias", "input_blocks.10.1.norm.weight", "input_blocks.10.1.norm.bias", "input_blocks.10.1.qkv.weight", "input_blocks.10.1.qkv.bias", "input_blocks.10.1.proj_out.weight", "input_blocks.10.1.proj_out.bias", "input_blocks.11.1.norm.weight", "input_blocks.11.1.norm.bias", "input_blocks.11.1.qkv.weight", "input_blocks.11.1.qkv.bias", "input_blocks.11.1.proj_out.weight", "input_blocks.11.1.proj_out.bias", "input_blocks.13.0.skip_connection.weight", "input_blocks.13.0.skip_connection.bias".
Unexpected key(s) in state_dict: "input_blocks.15.0.in_layers.0.weight",
... [omitted for brevity]
"input_blocks.17.0.out_layers.3.weight", "input_blocks.17.0.out_layers....
size mismatch for input_blocks.0.0.weight: copying a param with shape torch.Size([128, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 3, 3, 3]).
size mismatch for input_blocks.0.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for input_blocks.1.0.in_layers.0.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for input_blocks.1.0.in_layers.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for input_blocks.1.0.in_layers.2.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
... [omitted for brevity]
I would be happy to take a stab at this if you have any ideas. But i'm not entirely sure this request even makes sense as I'm still learning about the diffusion model and how it works. Is it possible to have it generate smaller images? (and ultimately reduce the memory footprint of the model).
If you have other ideas of how to reduce vram usage i would be interested in hearing that as well / discussing further!
Justin,
So many thanks for developing this version of vqgan art generator. I thought I'd send you a note on a persistent problem I'm having. I can't load vqgan models - wikiart_16384 & coco off the collab notebook.
Should I run this off Linux for better reliability?
Thanks again!
After running the Colab once, on the second try, I get the message that CUDA is out of memory. What is the best practice to reset this? I've tried a few different solutions from StackOverflow and forums, but none that has worked.
Hi Justin, I'm getting an SSL error after running the VQGAN+CLIP module. I checked the URL and it appears that forcing http results in a download, so maybe pytorch.org let their SSL cert slide? I tried to figure out if I could change model = load_vqgan_model(args.vqgan_config, args.vqgan_checkpoint).to(device)
to spit out an http URL instead of https, but I don't really know enough Python to help. Part of the error message:
Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth"
...
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)
When trying to run this it comes with the issue of torch not being defined and as some one who knows little to nothing of code I do not know how to fix this. Many thanks for any help.
torch.cuda.empty_cache()
with torch.no_grad():
torch.cuda.empty_cache()
I'm unable to render a video based on the frames made while having Diffusion checked, seems that each frame is replacing the former.
Hi there,
I'm using Colab Pro to do some ML experiments and often fail to initialise the RN50x4 CLIP model. Sometimes I even have trouble getting the RN101 model to load up. RN50x16 has never worked in my experience. The notebook is running on a P100 GPU and mentions that "x4 and x16 models for CLIP may not work reliably on lower-memory machines".
I'm just wondering if I need an even more capable GPU (in terms of VRAM) or if there is some problem with the code? I'm not an expert with Tensorflow/ML so apologies if there's a simple solution to this.
Output:
Executing using VQGAN+CLIP method
Using device: cuda:0
Using text prompt: ['Happy flowers in a sunlit field']
Using image prompts: ['/content/test.jpg']
Using seed: 5476245293527641887
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
[<ipython-input-41-541d444115c2>](https://localhost:8080/#) in <module>()
77 print('Using seed:', seed)
78
---> 79 model = load_vqgan_model(args.vqgan_config, args.vqgan_checkpoint).to(device)
80 perceptor = clip.load(args.clip_model, jit=False)[0].eval().requires_grad_(False).to(device)
81
NameError: name 'load_vqgan_model' is not defined
Output:
Executing using VQGAN+CLIP method
Using device: cuda:0
Using text prompt: ['Happy flowers in a sunlit field']
Using image prompts: ['/content/test.jpg']
Using seed: 5476245293527641887
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
[<ipython-input-41-541d444115c2>](https://localhost:8080/#) in <module>()
77 print('Using seed:', seed)
78
---> 79 model = load_vqgan_model(args.vqgan_config, args.vqgan_checkpoint).to(device)
80 perceptor = clip.load(args.clip_model, jit=False)[0].eval().requires_grad_(False).to(device)
81
NameError: name 'load_vqgan_model' is not defined
when i try to use ISR Execution that error pops up... also if i try to use Generate a video after that error i get this error FileNotFoundError: [Errno 2] No such file or directory: '/content/vqgan-steps/0009.png'......... am i doing something wrong?
I just started seeing this error when running this notebook on Colab recently, during the "Load libraries and definitions" block. Here's a screenshot...
I've run hundreds of cycles of this notebook in the past, but within the last week or two, it started failing every time.
I'm not an experienced python dev, so I'm a bit out of my element, but I can help troubleshoot from my side if that helps!
FileNotFoundError: [Errno 2] No such file or directory: 'video.mp4'
I thought ESRGAN was simply inverting the colors of the original but I took the image results of ESRGAN into a graphics program and this isn't the case. Hue is shifted in parts of the image, and maintained in other parts.
Examples attached. The original is in blue and green, ESRGAN's result gave a red background but foreground elements are maintained somehow. I have my own drive folder that I upscale images from, but ESRGAN still changes the colors even if you change the target directory to '../ESRGAN/LR/' to use the given example images.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.