replicate / scribble-diffusion

Turn your rough sketch into a refined image using AI

🖍️ Scribble Diffusion

Try it out at scribblediffusion.com

How it works

This app is powered by:

🚀 Replicate, a platform for running machine learning models in the cloud.

🖍️ ControlNet, an open-source machine learning model that generates images from text and scribbles.

▲ Vercel, a platform for running web apps.

⚡️ Next.js server-side API routes, for talking to the Replicate API.

👀 Next.js React components, for the browser UI.

🍃 Tailwind CSS, for styles.

Development

1. Install a recent version of Node.js
2. Copy your Replicate API token and set it in your environment:

   echo "REPLICATE_API_TOKEN=<your-token-here>" > .env.local
   

3. Install dependencies and run the server:

   npm install
   npm run dev
   

4. Open localhost:3000 in your browser. That's it!

scribble-diffusion's Issues

let me start clicking outside the canvas to draw inside the canvas (if possible)

500 Server Error!!

POST http://localhost:3000/api/predictions 500 (Internal Server Error), after clicking on GO,
It looks like they blocked de API

Mi infancia

Mi infancia fue como un libro de aventuras, lleno de páginas rebosantes de risas, descubrimientos y momentos inolvidables. Crecí en un pequeño pueblo rodeado de naturaleza, donde cada día era una nueva aventura esperando a ser explorada.
Recuerdo los largos días de verano jugando al aire libre con mis amigos, construyendo fortalezas en los árboles y persiguiendo mariposas por los prados. Las risas resonaban en cada rincón mientras inventábamos juegos y desafíos que nos mantenían entretenidos durante horas.

Pero recuerdo más el día que estábamos con ms amigos jugando a las escondidas, entre eso Sali corriendo a esconderme cuando no me percate que había una ventana abierta, me colgué super fuerte la cabeza, salió sangre, corrí para donde mis papas, fue un susto horrible pero mi mama estaba allí para mí, siempre lo está.

make predictions shareable

Do OpenGraph images work on twitter?

I posted a link to a Scribble on twitter and it didn't "unfurl"

Maybe Twitter expects a different format for OG meta tags?

record a demo video

• How to use it
• README
• ControlNet repo
• Replicate model
• React frontend
• Upload sketches to upload.io
• Node.js backend talking to Replicate's API
• What next? Share a link, sync localStorage, OpenGraph images...

Whole page moving randomly on android

When trying to draw something on android (be it using finger on the phone, or using stylus on tablet) at random moment the whole page moves with finger or stylus. Usually up or down, as it exceeds the screen in this direction.

This means I can't draw with any precision.

It happens almost every time I try to draw a circle.

REPLICATE_API_TOKEN not found

REPLICATE_API_TOKEN has been configured, but it still shows "The REPLICATE_API_TOKEN environment variable is not set. See README.md for instructions on how to set it."

make prompt required before form can be submitted

retrieve old scribbles from scribblediffusion.com (03/2023) error 500

hello,
unfortunately i have only saved links to some (11) scribbles done on scribblediffusion.com, without saving the drawing and the output.
have they been wiped? any chance to retrieve them providing the links? they are from 03/2023.

regards

Cat

Black

Sola

Sola og elv

Using Scribble Diffusion with Automatic1111

Hello. I think it would be great to run this with automatic1111. Doesn't need internet connection and is convenient

Erase Tool would be interesting

A tool to erase the scribbles would be interesting to retouch the drawing with each new generation.

Не можливо малювати

Доброго дня
Зовсім не зрозумілий зовнішній вигляд. Програмні коди, реклама. Зовсім не має віконця, щоб малювати. Нічого не розумію. Скрін додаю

starting scribbles are wrong size on iOS

The scribbles get chopped off, presumably because the canvas is narrower than the expected width of 512

Integration in stable diffusion

I dont want to use the "replicate" network.
So i tried to integrate "scribble diffusion" in the interface of automatic1111.
Now i have "scribble diffusion" listed as an extension, but i dont see any extra menu or extra feature.

This is very new for me, so a couple of question:

• Does it make sense to try to integrate scribble diffusion is automatic1111 ?
• Should it work right now?
• Is there a way to use scribble diffusion without using the replicate network ?

Thank you for your time.

Allow uploading custom scribbles

It would be useful to upload your own scribble instead of using the current editor

Beautiful house and nature

Uploading 2302.05543.pdf…

Invalid webhook URL

• error node_modules\replicate\lib\predictions.js (20:12) @ Replicate.createPrediction
• error Invalid webhook URL
  null

Docker image ?

Is it imaginable to have a docker image for this app ?

on the small screen the footer text is going out and it's not make the container that wrap the text to be responsive

To make the footer text responsive, you can use CSS media queries to adjust the font size, line height, and container width based on the screen size. You can also use CSS flexbox or grid layout to ensure that the container wraps the text properly.

if need the css3 code

`/* Set a default font size and line height */
.footer-text {
font-size: 16px;
line-height: 1.5;
}

/* Use media queries to adjust font size and line height for smaller screens */
@media screen and (max-width: 600px) {
.footer-text {
font-size: 14px;
line-height: 1.3;
}
}

/* Use flexbox or grid to ensure container wraps text */
.footer-container {
display: flex;
flex-wrap: wrap;
max-width: 100%;
}
`

store state in localstorage

How to use mongo DB with it and store generated images in that

createPrediction error

It runs well until I click the image generation button.
Does anyone have a clue?
Here is the error message.

npm run dev

[email protected] dev
next dev

• ready started server on 0.0.0.0:3000, url: http://localhost:3000
• info Loaded env from C:\MyDevWork\scribble-diffusion.env.local
  (node:58892) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
  (Use node --trace-warnings ... to show where the warning was created)
  (node:58892) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
• event compiled client and server successfully in 875 ms (205 modules)
• wait compiling...
• event compiled client and server successfully in 325 ms (205 modules)
• wait compiling / (client and server)...
• event compiled client and server successfully in 487 ms (1134 modules)
• wait compiling /api/predictions (client and server)...
• event compiled successfully in 66 ms (67 modules)
• error node_modules\replicate\lib\deployments.js (21:12) @ Replicate.createPrediction
• error Invalid webhook URL
  null
• wait compiling /_error (client and server)...
• event compiled client and server successfully in 129 ms (1136 modules)

Use a new canvas implementation

The existing react-sketch-canvas package that's currently used has a number of issues:

• #17
• #15

There are probably a lot of alternative npm packages out there that do the same thing, but better. Here are some that might be useful:

• https://www.npmjs.com/package/react-canvas-draw
• https://github.com/federico-moretti/canvas-free-drawing
• https://github.com/rough-stuff/rough
• https://github.com/pshihn/path-data-parser
• https://github.com/pshihn/bezier-points
• https://github.com/pshihn/points-on-path

Horse

Tasks

Beta Give feedback

1. Cat #78
   +0
2. retrieve old scribbles from scribblediffusion.com (03/2023) error 500 #76
   +0
3. Create $ #75
   +0
4. error API request failed: Unprocessable Entity #68
   +0

Canvas doesn't work properly on Firefox / Android

I tried https://scribblediffusion.com/ on multiple devices (phone, tablet) with Mozilla Firefox, and drawing doesn't work properly:

• the initial image is not fully rendered: it looks like the image is randomly cropped on the right and on the bottom
• when drawing on a touch screen drawing doesn't follow the finger correctly. Well, sometimes it works, sometimes not.

Warning: A title element received an array with more than 1 element as children

In the Node.js console in development:

Warning: A title element received an array with more than 1 element as children. In browsers title Elements can only have Text Nodes as children. If the children being rendered output more than a single text node in aggregate the browser will display markup and comments as text in the title and hydration will likely fail and fall back to client rendering
    at title
    at head
    at Head (webpack-internal:///./node_modules/next/dist/pages/_document.js:279:1)
    at html
    at Html (webpack-internal:///./node_modules/next/dist/pages/_document.js:678:104)
    at Document

Not able to set it up locally

Hey,

I have been wanting to set up Scribble Diffusion locally and followed all steps for it and even added my Replicate API Key into .env.local file but as you can see from the Screenshot that Its failing to recognize the API Key's presence even though it exists.

What should i do?

Option for Uploading an Existing Scribble

This is awesome!

One thing that I think is missing is giving the option to upload my existing scribble as an image.

Using existing art would be very helpful rather than drawing from scratch on the canvas.

Unhandled Runtime Error

UploadApiError: Unauthenticated: the domain 'http://192.168.1.101:3000' has not been whitelisted by this account.

Error on replication on ubuntu server

When I load sribble broadcast from my server it gives me this error:

Then trying to generate the image gives me other errors.

Only allowing about 10 minutes of use

We are using Scribblediffusion.com with elementary school students in a computer lab setting. The first group was able to play for about 10 minutes before the waiting bubbles just continue without creating an image. The next group ran into the waiting bubbles immediately upon trying their first image. This has been consistent over 2 days of class.

Is there a limit to the amount of time a particular device can access the site?

Can't deplogy on my server.

Clases de matematicas

Es sobre la clase de matematicas

Tasks

Beta Give feedback

No tasks being tracked yet.

add OpenGraph image

Trying to build locally or through cloudflare. Fails to compile

I try to build running command "npm run build"

It tries to run through for a few minutes but then errors out with;

info - Creating an optimized production build . [webpack.cache.PackFileCacheStrategy] Caching failed for pack: Error: Unable to snapshot resolve dependencies
info - Creating an optimized production build
Failed to compile.

Error: EISDIR: illegal operation on a directory, readlink 'directory/scribble-diffusion-main\pages_app.js'

The REPLICATE_API_TOKEN environment variable is not set

i have checked that my token is config in .env.local file,where is the promble?

error API request failed: Unprocessable Entity

It was executed in the M1 environment. I proceeded as written in the ReadMe file.
When I press the Go Button in , the following error occurs.

• ready started server on 0.0.0.0:3000, url: http://localhost:3000
• info Loaded env from /Users/ojunsol/development/scribble-diffusion/.env.local
• event compiled client and server successfully in 392 ms (205 modules)
• wait compiling...
• event compiled client and server successfully in 215 ms (205 modules)
• wait compiling / (client and server)...
• event compiled client and server successfully in 325 ms (1134 modules)
• wait compiling /api/predictions (client and server)...
• event compiled successfully in 75 ms (65 modules)
• wait compiling /_error (client and server)...
• error node_modules/replicate/index.js (162:12) @ Replicate.request
• error API request failed: Unprocessable Entity
  null

Use real paper scribbles

Hi,
Thanks for creating this. It really sparked the creativity for me and my kids. I could not resist thinking about adding the paper scribbels functionality to this.
(if this is not supposed to be a ticket, certainly close it and point me to the right direction.

I have created a fork: Paper sketch diffusion.
https://github.com/woutersf/paper-sketch-diffusion

The idea is to allow your input to also allow a picture of a paper sketch.

Therefor I added a button (not styled yet)

Which then promps the camera permission:

And you are allowed to take a picture of a sketch.

I also transform the sketch to black and white and threshold it (black or white contrast) so that it is "scribblable" .

I then want this image data to flow into the scribble (see red arrow above), This is where I'm stuck now.
I would like to know how to flow the image data I have and not use the canvas that was hand drawn.
You will see in my code that I have no clue about React.
woutersf@14a5f47

You can easily try it out since you don't even need the replicate key to get this working.
Any help is much appreciated.

Unable to generate image

Keep loading, check network request and find upload error:

Unable to generate image in Safari

Keep loading after Go button clicked, check network request and find predictions error with status 500 in Safari.

but works fine in Chrome on the same computer.

Unhandled Runtime Error

I got this error when running the project on local

UploadApiError: Upload limit reached for this month! Please upgrade at https://upload.io/