Comments (5)
I just merged some very basic support for ControlNet via #68 . The weight file data/controlnet.safetensors
can be obtained from the HuggingFace hub here.
Note that this is for the "canny" version of ControlNet and the edge detection part has to be done before submitting the input image (I plan on fixing this soonish). Here is how to run it:
cargo run --example controlnet --features clap -- --cpu all --prompt "disco dancer with colorful lights" --input-image vermeer_canny_edged.png
The input image vermeer_canny_edged.png
looked like:
from diffusers-rs.
Cool! I learn a lot from this feature PR. Thanks!
I will find some free time to add other ControlNet capabilities.
from diffusers-rs.
Great that you find it helpful, I've also included the edge detection bit now (through an external crate) so there is no need to pre-process the original image anymore. There are also some specific instructions in the readme file.
from diffusers-rs.
Would absolutely love to use ControlNet scribble here!
from diffusers-rs.
If I wanted to stack multiple controlnets together (depth, canny, normal) for example, how difficult would you estimate this to be to implement @LaurentMazare?
from diffusers-rs.
Related Issues (20)
- Feature Request: Negative prompts HOT 1
- Add Scheduler trait/enum HOT 2
- m1 mac gpu HOT 6
- Google Colab Notebook to run diffusion experiment on the GPU
- Embed the examples logic into the pipeline HOT 1
- How to load a parameter file in safetensors format? HOT 1
- PytorchStreamReader failed reading zip archive HOT 2
- Bad distorted picture using the in-painting example provided HOT 4
- Loading of text embeddings in pt format? HOT 2
- Example of inpaint doesn't work for Stable Diffusion 2.1 HOT 2
- CUDA out of memory on 12GB GPU HOT 2
- Error: The system cannot find the file specified. (os error 2) HOT 2
- Tracking issue for SD ecosystem feature parity HOT 6
- DirectML Support HOT 1
- Cannot link when used together with cxx-qt crate HOT 1
- CUDA/GPU Not Working. HOT 1
- STATUS_DLL_NOT_FOUND HOT 1
- Benchmarks? HOT 1
- Integration with Stable Diffusion XL 1.0 ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffusers-rs.