machineminded / fooocus-inswapper Goto Github PK

View Code? Open in Web Editor NEW

This project forked from lllyasviel/fooocus

69.0 1.0 14.0 34.25 MB

Focus on prompting and generating with an inswapper integration

License: GNU General Public License v3.0

JavaScript 2.44% Python 96.93% CSS 0.35% Jupyter Notebook 0.04% Batchfile 0.05% Shell 0.14% Dockerfile 0.05%

fooocus-inswapper's Introduction

Fooocus-inswapper

This is a fork of Fooocus. This fork integrates the following:

Insightface/inswapper library used by roop, ReActor, and others
PhotoMaker based on 🤗 diffusers
InstantID based on 🤗 diffusers

The goal of this repository is to stay up-to-date with the main repository, while also maintaining the above integrations.

For more detailed and official documentation, please refer to lllyasviel's repository.

A standalone installation does not exist for this repository.

Installation (Windows)

The installation assumes CUDA 11.8. If you need a different version, please update configure.bat with the correct URL to the desired CUDA version.

Ensure Microsoft Visual C++ Redistributable is installed.
Enusre Microsoft Visual C++ Build Tools are installed. Install the Desktop workload.
Run git clone https://github.com/machineminded/Fooocus-inswapper.git
Execute configure.bat

Inswapper Usage

Inswapper will activate if "Enabled" is checked.

Execute run.bat if you're on Windows, or run.sh if you're on a Linux-based OS.

how-to.mp4

PhotoMaker Usage

In this fork, PhotoMaker utilizes 🤗 diffusers, so it runs outside of the ksampler pipelines. I'd like to eventually add inpainting and ControlNet for 🤗 diffusers but it will take some time. Keep in mind that PhotoMaker currently requires 15GB of VRAM! The following Fooocus configuration items are passed to the PhotoMaker 🤗 diffusers pipeline:

Resolution (width and height)
Prompt (and generated prompts from selected styles)
Negative Prompt (and generated prompts from selected styles)
Steps
CFG/Guidance Scale
Seed
LoRAs
Sampler (not fully implemented)
Scheduler (not fully implemented)

PhotoMaker General Usage

Navigate to the PhotoMaker tab.
Click "Enable"
Load images from your PC.
Enter your prompt and be sure to include "man img" or "woman img" depending on the subject at hand. img in the prompt is expected by PhotoMaker.
Click "Generate"

Experiment with also adding an image to the Inswapper tab to overlay the generated image.

Note: Unchecking "Enable" will unload the PhotoMaker pipeline from memory!

PhotoMaker LoRA Usage

Select the LoRAs you want to use as usual.
Navigate to the PhotoMaker tab.
Click "Enable" then click "Generate"

If you change the LoRAs or their weights:

Uncheck "Enabled" to unload the model from memory
Re-check "Enabled" and click "Generate" to reload the LoRAs and the pipeline into memory.

Supported PhotoMaker samplers

euler
euler ancestral
DPM++ 2M SDE
DPM++ 2M SDE Karras
Will default to DDIM Scheduler for anything else

InstantID Usage

In this fork, InstantID utilizes 🤗 diffusers, so it runs outside of the ksampler pipelines. I'd like to eventually add inpainting and ControlNet for 🤗 diffusers but it will take some time. This requires high amounts of VRAM (easily 18GB or more). The following Fooocus configuration items are passed to the InstantID 🤗 diffusers pipeline:

Resolution (width and height)
Prompt (and generated prompts from selected styles)
Negative Prompt (and generated prompts from selected styles)
Steps
CFG/Guidance Scale
Seed
LoRAs
Sampler (not fully implemented)
Scheduler (not fully implemented)

InstantID General Usage

Navigate to the InstantID tab.
Click "Enable"
Load images from your PC.
Enter your prompt and be sure to include "man img" or "woman img" depending on the subject at hand. img in the prompt is expected by PhotoMaker.
Click "Generate"

Experiment with also adding an image to the Inswapper tab to overlay the generated image.

Note: Unchecking "Enable" will unload the InstantID pipeline from memory!

InstantID LoRA Usage

Select the LoRAs you want to use as usual.
Navigate to the InstantID tab.
Click "Enable" then click "Generate"

If you change the LoRAs or their weights:

Uncheck "Enabled" to unload the model from memory
Re-check "Enabled" and click "Generate" to reload the LoRAs and the pipeline into memory.

Supported InstantID samplers

euler
euler ancestral
DPM++ 2M SDE
DPM++ 2M SDE Karras
Will default to DDIM Scheduler for anything else

Colab

(Not fully working yet)

Colab	Info
	Fooocus Official

Issues

Please report any issues in the Issues tab. I will try to help as much as I can.

To Do

🚀 Allow changing of insightface parameters (Inswapper)
🚀 Allow customizable target image (Inswapper)
🚀 Increase diffusers pipeline to > 77 tokens (PhotoMaker)
🚀 Allow dynamic loading of LoRAs into diffusers pipeline (PhotoMaker)

fooocus-inswapper's People

Contributors

Stargazers

Watchers

Forkers

weyjei learncodepractice codingjourneyfromunemployment dddmmm5 p971607 rayochoajr rokonezoko55555555 samuraibarbi radicalz1 testujemai render-ai thrddr paulofilip3 korsar011

fooocus-inswapper's Issues

[Feature Request]: Juggernaut 9 is out now

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits

What would your feature do?

JuggernautXL V9 is out now but 8 is required to run the application.

Proposed workflow

Suggestion to either change the auto download to v9 or give users the option to download 8 or not, we don't need 8 if we have 9.

Additional information

No response

ImportError: Unable to import 'IOComponent' from 'gradio.components.base'

python .\launch.py

[System ARGV] ['.\launch.py']
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.865
Traceback (most recent call last):
File "C:\Users\mrdie\Documents\FocusImswapper\Fooocus-inswapper\launch.py", line 128, in
from webui import *
File "C:\Users\mrdie\Documents\FocusImswapper\Fooocus-inswapper\webui.py", line 14, in
import modules.gradio_hijack as grh
File "C:\Users\mrdie\Documents\FocusImswapper\Fooocus-inswapper\modules\gradio_hijack.py", line 21, in
from gradio.components.base import IOComponent, _Keywords, Block
ImportError: cannot import name 'IOComponent' from 'gradio.components.base' (C:\Users\mrdie\Documents\FocusImswapper\Fooocus-inswapper\venv\Lib\site-packages\gradio\components\base.py)

osx installation [newbie]

Read Troubleshoot

[x] I confirm that I have read the Troubleshoot guide before making this issue.

Describe the problem
thank, i've followed the installations steps but the Inswapper tab doesn't appear:

Full Console Log

ERROR: Could not find a version that satisfies the requirement onnxruntime-gpu==1.17.0 (from versions: none)
ERROR: No matching distribution found for onnxruntime-gpu==1.17.0

DEPRECATION: torchsde 0.2.5 has a non-standard dependency specifier numpy>=1.19.*; python_version >= "3.7". pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of torchsde or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063

(base) silvio@Silvios-iPro Fooocus-inswapper % .\venv\Scripts\activate     
zsh: command not found: .venvScriptsactivate

Feature: Build a run.bat file when configure.bat is completed successfully

Is your feature request related to a problem? Please describe.

Instead of having users having to write a command each time they want to run it or create their own bat file, it should be autogenerated when configure.bat is ran successfully

Describe the idea you'd like

create a run.bat file with the follow inside. Unsure if that will work for all OS though as I only have experience with windows
.\venv\Scripts\activate && python launch.py

Save images in output as jpeg

This will update the code to save output as both png and jpeg. Code is basically ready in feature/jpeg-save.

History Log does not work

Read Troubleshoot

[x] I confirm that I have read the Troubleshoot guide before making this issue.

Describe the problem
Windows 10
Firefox/Chrome

When using History Log (Advanced -> History Log), the browser returns this error instead {"detail":"File not allowed: //desk1/G/bit/SD/images/2024-02-19/log.html."}

Couple of caveats.

The link is to a mapped drive, but a valid drive. If I use file://///desk1/G/bit/SD/images/2024-02-19/log.html this will open the log html file.
In my installation folder, I have used a symlink to file://///desk1/G/bit/SD/images/ instead of C:\Fooocus inswapper\Fooocus-inswapper

Unsure if either of the above caused the issue, or if this is prevalent on a normal install that didn't muck with the output folder.

Full Console Log

browser error (chrome and firefox)
{"detail":"File not allowed: //desk1/G/bit/SD/images/2024-02-19/log.html."}

Integrate InstantID

Fairly straightfoward: Integrate InstantID

Installation issues

Read Troubleshoot

[x] I admit that I have read the Troubleshoot before making this issue.

Hello MachineMinded, I'm having some issues getting everything installed with this.
Can you tell the correct initial install procedure as far as which python/bat scripts to run and I will let you know where I'm getting stuck.

InstantID Inpainting

Some code has already been checked in, but needs to be updated to use the provided pose image.

instantid-integration problem(s)

had trouble installing at first from this link
https://github.com/machineminded/Fooocus-inswapper/tree/feature/instantid-integration
because instruction say to git clone git clone https://github.com/machineminded/Fooocus-inswapper.git
which I install a version without instantid tab
got it working never the less

so i create a new folder name "Fooocus-inswapper-feature-instantid-integration"
this time instead of git clone, I zip the files and download
with this new instalation I see instantid tab

tab 1 inswappers works. athought I gets errors in CMD window in red color text:

2024-02-15 15:56:28.5015420 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\Fooocus-inswapper-feature-instantid-integration\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

inswappers did works. Prompt "painting of a beautiful woman with long hair" and chose "Artstyle Waterolor"
results sort of look like her and is in watercolor style

moving on to tab 2 photomaker
add "img, " to the begining of previous prompt. "img, painting of a beautiful woman with long hair" and choose "Artstyle Waterolor"
results worked, they look like a painting BUT not a watercolor, more like oil paint. So photomaker did not used "Artstyle Waterolor"

while at photomaker I tried uploading 4 images of the same girl. It works similar to uploading just 1 image

move on to tab 3 instantid
remove "img, " from prompt, choose "Artstyle Waterolor" upload 1 image
got ERROR

InstantID: Begin
InstantID: Positive prompt: watercolor painting painting of a beautiful woman with long hair . vibrant, beautiful, painterly, detailed, textural, artistic
InstantID: Negative prompt: anime, photorealistic, 35mm film, deformed, glitch, low contrast, noisy
InstantID: Loading diffusers pipeline into memory.
Traceback (most recent call last):
  File "E:\Fooocus-inswapper-feature-instantid-integration\modules\async_worker.py", line 885, in worker
    handler(task)
  File "E:\Fooocus-inswapper-feature-instantid-integration\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "E:\Fooocus-inswapper-feature-instantid-integration\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "E:\Fooocus-inswapper-feature-instantid-integration\modules\async_worker.py", line 810, in handler
    imgs = generate_instantid(instantid_image_path, instantid_pose_image_path, instantid_prompt, instantid_negative_prompt, steps, task['task_seed'], width, height, guidance_scale, loras, sampler_name, scheduler_name, async_task, instantid_identitynet_strength_ratio, instantid_adapter_strength_ratio)
  File "E:\Fooocus-inswapper-feature-instantid-integration\modules\instantid.py", line 144, in generate_instantid
    pipe = load_model(loras, sampler_name)
  File "E:\Fooocus-inswapper-feature-instantid-integration\modules\instantid.py", line 50, in load_model
    app = FaceAnalysis(name='antelopev2', root='InstantID', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
  File "E:\Fooocus-inswapper-feature-instantid-integration\venv\lib\site-packages\insightface\app\face_analysis.py", line 43, in __init__
    assert 'detection' in self.models
AssertionError
Total time: 0.98 seconds

Fix an issue with multiple faces

Describe the problem
If multiple faces are present on the output image, the face swap logic fails.

Either we need to allow picking which face to swap, or just pick the first face. This bug sort of coincides with allowing changing of the inswapper parameters, so I'll probably just fix the bug by allowing the user to pick which face to swap based on index similar to roop and ReActor.

stuck at "Waiting for task to start ..."

Read Troubleshoot

[x] I confirm that I have read the Troubleshoot guide before making this issue.

Describe the problem
A clear and concise description of what the bug is.

Full Console Log
Paste the full console log here. You will make our job easier if you give a full log.

using:

!pip install pygit2==1.12.2
%cd /content
!git clone https://github.com/machineminded/Fooocus-inswapper.git
!git checkout feature/issue-21-mult-target-insightface
%cd /content/Fooocus-inswapper
!chmod +x configure-colab.sh
!/content/Fooocus-inswapper/configure-colab.sh

%pip install -r requirements_versions.txt

!python launch.py --theme dark --share --disable-offload-from-vram --always-high-vram --all-in-fp16

EP ERROR

Read Troubleshoot

[x] I confirm that I have read the Troubleshoot guide before making this issue.

Describe the problem
*************** EP Error ***************

Full Console Log
:\Fooocus-inswapper-main\ldm_patched\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [01:24<00:00, 10.54s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.94 seconds
Inswapper: Target index: 0.0
download_path: ./checkpoints\models\buffalo_l
Downloading ./checkpoints\models\buffalo_l.zip from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 281857/281857 [01:33<00:00, 3000.07KB/s]
2024-02-22 09:33:38.8141340 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

*************** EP Error ***************
EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:456 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

2024-02-22 09:33:39.1353830 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
2024-02-22 09:33:39.9811181 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

2024-02-22 09:33:40.0300656 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
2024-02-22 09:33:40.1010470 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

2024-02-22 09:33:40.1605064 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
2024-02-22 09:33:40.4148663 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

2024-02-22 09:33:40.4720848 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
2024-02-22 09:33:40.5211007 [E:onnxruntime:Default, provider_bridge_ort.cc:1532 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

2024-02-22 09:33:40.6631070 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
2024-02-22 09:33:42.1722514 [E:onnxruntime:Default, provider_bridge_ort.cc:1546 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 885, in worker
handler(task)
File "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Users\moath alhssah\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus-inswapper-main\modules\async_worker.py", line 837, in handler
imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
File "C:\Fooocus-inswapper-main\modules\face_swap.py", line 17, in perform_face_swap
from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
File "C:\Fooocus-inswapper-main\inswapper\restoration.py", line 10, in
from basicsr.utils import imwrite, img2tensor, tensor2img
ModuleNotFoundError: No module named 'basicsr'
Total time: 464.33 seconds

[Bug]: Inswapper does not fail gracefully if there is no image selected for source face image

Checklist

The issue has not been resolved by following the troubleshooting guide
The issue exists on a clean installation of Fooocus
The issue exists in the current version of Fooocus
The issue has not been reported before recently
The issue has been reported before but has not been fixed yet

What happened?

If someone does not enter a face image into "Source Face Image" the script will not fail gracefully. It will error on the backend,

Steps to reproduce the problem

Open inswapper
Enable
Do not enter a face image

What should have happened?

simply ignore inswapper when user doesn't enter a face image
Do not start generation and show an error message

What browsers do you use to access Fooocus?

No response

Where are you running Fooocus?

None

What operating system are you using?

No response

Console logs

File "C:\Fooocus inswapper\Fooocus-inswapper\modules\async_worker.py", line 1047, in worker
    handler(task)
  File "C:\Fooocus inswapper\Fooocus-inswapper\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus inswapper\Fooocus-inswapper\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Fooocus inswapper\Fooocus-inswapper\modules\async_worker.py", line 959, in handler
    imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_source_image_indicies, inswapper_target_image_indicies)
  File "C:\Fooocus inswapper\Fooocus-inswapper\modules\face_swap.py", line 12, in perform_face_swap
    source_image = Image.fromarray(inswapper_source_image)
  File "C:\Fooocus inswapper\Fooocus-inswapper\venv\lib\site-packages\PIL\Image.py", line 2943, in fromarray
    arr = obj.__array_interface__
AttributeError: 'NoneType' object has no attribute '__array_interface__'
Total time: 22.46 seconds

Additional information

No response

InstantID - "Waiting for task to start ..."

Using InstandID for the first time, just getting "Waiting for task to start ..." and don't think anything else is happening.

"No module named basicsr"

When using InSwapper for the first time:

"Target Image Index" set to -1

**ModuleNotFoundError: No module named 'basicsr'****

Per ChatGPT4:

The error message you're encountering, ModuleNotFoundError: No module named 'basicsr', indicates that the Python environment you're working in does not have the basicsr module installed, which is required for the execution of your code. The basicsr library is a part of Basic Super-Resolution Toolbox, commonly used for image and video super-resolution tasks.

To resolve this issue, you need to install the basicsr package. You can usually do this via pip, Python's package manager. Open your terminal or command prompt and activate your Python environment where you're running the code. Then, install basicsr using the following command:

bash

pip install basicsr

After installation, try running your script again. The ModuleNotFoundError should be resolved, and your code should proceed beyond the point where it was previously failing.

If you encounter any other ModuleNotFoundError errors for different packages in the future, you can use the same approach to install the missing modules.

"PhotoMaker currently does not support multiple trigger words in a single prompt."

Settings are for PhotoMaker:
Enabled checked
1 image input
prompt = holding a chainsaw, horror film, zombie apocalypse

File "C:\Fooocus-inswapper-main\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
  return func(*args, **kwargs)
File "C:\Fooocus-inswapper-main\photomaker\pipeline.py", line 344, in __call__
  ) = self.encode_prompt_with_trigger_word(
File "C:\Fooocus-inswapper-main\photomaker\pipeline.py", line 176, in encode_prompt_with_trigger_word
  raise ValueError(
ValueError: PhotoMaker currently does not support multiple trigger words in a single prompt.                            
Trigger word: img, Prompt: cinematic still holding a chainsaw, horror film, zombie apocalypse . emotional, harmonious, vignette, 4k epic detailed, shot on kodak, 35mm photo, sharp focus, high budget, cinemascope, moody, epic, gorgeous, film grain, grainy.
Total time: 25.02 seconds

Allow a custom target image for inswapper

Instead of using the AI generated image, we should allow uploading of a custom image.

InstantID LCM Support

Need to pass LCM values from Fooocus to InstantID and configure as such.

errors errors

is there anyone actually used it like only inswapper option atleast and it worked?

Add metadata tags to image based on mashb1t fork

Need to check out mashb1t's fork and integrate how they are saving metadata into the image.

Proposal for Implementing New Upscaling Model

I hope this message finds you well. I am writing to propose the implementation of the bdsqlsz_controlllite_xl_tile_realistic upscaling model to Fooocus-inswapper project released a week ago.

This model presents promising capabilities for enhancing image quality

Looking forward to your feedback.

Best regards
thanks in advance

[https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite]

Ability to cancel diffusers pipelines

PhotoMaker and InstantID both use 🤗 diffusers, and we need an ability to interrupt the pipeline during generation.

Please review: https://huggingface.co/docs/diffusers/main/en/using-diffusers/callback#interrupt-the-diffusion-process

Add preview images for PhotoMaker

Since I am using diffusers for PhotoMaker, I need to figure out a way to generate the gallery preview images using the pipeline's callback. Some code is already present in the project. Currently it will output progress images to the gradio temp folder, I need to figure out how to hook them back into the Fooocus preview gallery component.

[Bug]: Windows - ModuleNotFoundError: No module named 'diffusers'

Checklist

The issue has not been resolved by following the troubleshooting guide
The issue exists on a clean installation of Fooocus
The issue exists in the current version of Fooocus
The issue has not been reported before recently
The issue has been reported before but has not been fixed yet

What happened?

After successful installation, I get the error on the first launch:

ModuleNotFoundError: No module named 'diffusers'

Steps to reproduce the problem

git clone
configure.bat
run.bat

What should have happened?

I expected the GUI of the application to start

What browsers do you use to access Fooocus?

No response

Where are you running Fooocus?

None

What operating system are you using?

Windows 11

Console logs

D:\WM>git clone https://github.com/machineminded/Fooocus-inswapper.git
Cloning into 'Fooocus-inswapper'...
...
Resolving deltas: 100% (3804/3804), done.

D:\WM\Fooocus-inswapper>configure.bat
Updated Git hooks.
Git LFS initialized.
Cloning into 'CodeFormer'...
remote: Enumerating objects: 300, done.
remote: Counting objects: 100% (98/98), done.
remote: Compressing objects: 100% (93/93), done.
remote: Total 300 (delta 57), reused 5 (delta 5), pack-reused 202 (from 1)
Receiving objects: 100% (300/300), 6.99 MiB | 14.44 MiB/s, done.
Resolving deltas: 100% (102/102), done.

... removed collecting and downloading step due to error: body is too long ...

Building wheels for collected packages: antlr4-python3-runtime, insightface, controlnet_aux, ffmpy, filterpy
  Building wheel for antlr4-python3-runtime (pyproject.toml) ... done
  Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144577 sha256=23d8b70de03faa9fde3ba91109bb0581d8cabe24ef10e4d9d136eb3d67ea8ebf
  Stored in directory: c:\users\adolfo\appdata\local\pip\cache\wheels\1a\97\32\461f837398029ad76911109f07047fde1d7b661a147c7c56d1
  Building wheel for insightface (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for insightface (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [215 lines of output]
      WARNING: pandoc not enabled
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\insightface
      copying insightface\__init__.py -> build\lib.win-amd64-cpython-311\insightface
      creating build\lib.win-amd64-cpython-311\insightface\app
      copying insightface\app\common.py -> build\lib.win-amd64-cpython-311\insightface\app
      copying insightface\app\face_analysis.py -> build\lib.win-amd64-cpython-311\insightface\app
      copying insightface\app\mask_renderer.py -> build\lib.win-amd64-cpython-311\insightface\app
      copying insightface\app\__init__.py -> build\lib.win-amd64-cpython-311\insightface\app
      creating build\lib.win-amd64-cpython-311\insightface\commands
      copying insightface\commands\insightface_cli.py -> build\lib.win-amd64-cpython-311\insightface\commands
      copying insightface\commands\model_download.py -> build\lib.win-amd64-cpython-311\insightface\commands
      copying insightface\commands\rec_add_mask_param.py -> build\lib.win-amd64-cpython-311\insightface\commands
      copying insightface\commands\__init__.py -> build\lib.win-amd64-cpython-311\insightface\commands
      creating build\lib.win-amd64-cpython-311\insightface\data
      copying insightface\data\image.py -> build\lib.win-amd64-cpython-311\insightface\data
      copying insightface\data\pickle_object.py -> build\lib.win-amd64-cpython-311\insightface\data
      copying insightface\data\rec_builder.py -> build\lib.win-amd64-cpython-311\insightface\data
      copying insightface\data\__init__.py -> build\lib.win-amd64-cpython-311\insightface\data
      creating build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\arcface_onnx.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\attribute.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\inswapper.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\landmark.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\model_store.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\model_zoo.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\retinaface.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\scrfd.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      copying insightface\model_zoo\__init__.py -> build\lib.win-amd64-cpython-311\insightface\model_zoo
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty
      copying insightface\thirdparty\__init__.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty
      creating build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\constant.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\download.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\face_align.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\filesystem.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\storage.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\transform.py -> build\lib.win-amd64-cpython-311\insightface\utils
      copying insightface\utils\__init__.py -> build\lib.win-amd64-cpython-311\insightface\utils
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d
      copying insightface\thirdparty\face3d\__init__.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\io.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\light.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\render.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\transform.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\vis.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      copying insightface\thirdparty\face3d\mesh\__init__.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\io.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\light.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\render.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\transform.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\vis.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      copying insightface\thirdparty\face3d\mesh_numpy\__init__.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh_numpy
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\morphable_model
      copying insightface\thirdparty\face3d\morphable_model\fit.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\morphable_model
      copying insightface\thirdparty\face3d\morphable_model\load.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\morphable_model
      copying insightface\thirdparty\face3d\morphable_model\morphabel_model.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\morphable_model
      copying insightface\thirdparty\face3d\morphable_model\__init__.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\morphable_model
      running egg_info
      writing insightface.egg-info\PKG-INFO
      writing dependency_links to insightface.egg-info\dependency_links.txt
      writing entry points to insightface.egg-info\entry_points.txt
      writing requirements to insightface.egg-info\requires.txt
      writing top-level names to insightface.egg-info\top_level.txt
      reading manifest file 'insightface.egg-info\SOURCES.txt'
      writing manifest file 'insightface.egg-info\SOURCES.txt'
      C:\Users\adolfo\AppData\Local\Temp\pip-build-env-7asd7ga0\overlay\Lib\site-packages\setuptools\command\build_py.py:215: _Warning: Package 'insightface.thirdparty.face3d.mesh.cython' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'insightface.thirdparty.face3d.mesh.cython' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'insightface.thirdparty.face3d.mesh.cython' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'insightface.thirdparty.face3d.mesh.cython' to be distributed and are
              already explicitly excluding 'insightface.thirdparty.face3d.mesh.cython' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      C:\Users\adolfo\AppData\Local\Temp\pip-build-env-7asd7ga0\overlay\Lib\site-packages\setuptools\command\build_py.py:215: _Warning: Package 'insightface.data.images' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'insightface.data.images' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'insightface.data.images' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'insightface.data.images' to be distributed and are
              already explicitly excluding 'insightface.data.images' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      C:\Users\adolfo\AppData\Local\Temp\pip-build-env-7asd7ga0\overlay\Lib\site-packages\setuptools\command\build_py.py:215: _Warning: Package 'insightface.data.objects' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'insightface.data.objects' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'insightface.data.objects' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'insightface.data.objects' to be distributed and are
              already explicitly excluding 'insightface.data.objects' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      creating build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core.cpp -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core.h -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core_cython.cpp -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      creating build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\Tom_Hanks_54745.png -> build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\mask_black.jpg -> build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\mask_blue.jpg -> build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\mask_green.jpg -> build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\mask_white.jpg -> build\lib.win-amd64-cpython-311\insightface\data\images
      copying insightface\data\images\t1.jpg -> build\lib.win-amd64-cpython-311\insightface\data\images
      creating build\lib.win-amd64-cpython-311\insightface\data\objects
      copying insightface\data\objects\meanshape_68.pkl -> build\lib.win-amd64-cpython-311\insightface\data\objects
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core_cython.c -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core_cython.cpp -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\mesh_core_cython.pyx -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      copying insightface\thirdparty\face3d\mesh\cython\setup.py -> build\lib.win-amd64-cpython-311\insightface\thirdparty\face3d\mesh\cython
      running build_ext
      building 'insightface.thirdparty.face3d.mesh.cython.mesh_core_cython' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for insightface
  Building wheel for controlnet_aux (pyproject.toml) ... done
  Created wheel for controlnet_aux: filename=controlnet_aux-0.0.7-py3-none-any.whl size=274371 sha256=6ba37889bdcc887366215a645553b9d8c00192056518a445c9fea7d065c6a922
  Stored in directory: c:\users\adolfo\appdata\local\pip\cache\wheels\08\a1\0f\79d1529bbc60d1598da66052a2c60a2a13e5ac462b5f990653
  Building wheel for ffmpy (pyproject.toml) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.2-py3-none-any.whl size=5606 sha256=332b9e5dd3ea659c74d27a117b33796d18fa1c06fdd2e1c2014870efbc49bf5c
  Stored in directory: c:\users\adolfo\appdata\local\pip\cache\wheels\55\3c\f2\f6e34046bac0d57c13c7d08123b85872423b89c8f59bafda51
  Building wheel for filterpy (pyproject.toml) ... done
  Created wheel for filterpy: filename=filterpy-1.4.5-py3-none-any.whl size=110542 sha256=45ee4cfa861abacb56ec5043612092e2020890d6cdf3523d15b11bf462041110
  Stored in directory: c:\users\adolfo\appdata\local\pip\cache\wheels\12\dc\3c\e12983eac132d00f82a20c6cbe7b42ce6e96190ef8fa2d15e1
Successfully built antlr4-python3-runtime controlnet_aux ffmpy filterpy
Failed to build insightface
ERROR: Could not build wheels for insightface, which is required to install pyproject.toml-based projects

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: python.exe -m pip install --upgrade pip
Looking in indexes: https://download.pytorch.org/whl/cu118
Collecting torch
  Downloading https://download.pytorch.org/whl/cu118/torch-2.3.1%2Bcu118-cp311-cp311-win_amd64.whl (2673.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 GB 3.0 MB/s eta 0:00:00
Collecting torchvision
  Downloading https://download.pytorch.org/whl/cu118/torchvision-0.18.1%2Bcu118-cp311-cp311-win_amd64.whl (4.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 15.0 MB/s eta 0:00:00
Collecting torchaudio
  Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.3.1%2Bcu118-cp311-cp311-win_amd64.whl (4.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/4.0 MB 16.9 MB/s eta 0:00:00
Collecting filelock (from torch)
  Downloading https://download.pytorch.org/whl/filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting typing-extensions>=4.8.0 (from torch)
  Downloading https://download.pytorch.org/whl/typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting sympy (from torch)
  Downloading https://download.pytorch.org/whl/sympy-1.12-py3-none-any.whl (5.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 16.7 MB/s eta 0:00:00
Collecting networkx (from torch)
  Downloading https://download.pytorch.org/whl/networkx-3.2.1-py3-none-any.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 11.7 MB/s eta 0:00:00
Collecting jinja2 (from torch)
  Downloading https://download.pytorch.org/whl/Jinja2-3.1.3-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 kB 7.7 MB/s eta 0:00:00
Collecting fsspec (from torch)
  Downloading https://download.pytorch.org/whl/fsspec-2024.2.0-py3-none-any.whl (170 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 170.9/170.9 kB 10.0 MB/s eta 0:00:00
Collecting mkl<=2021.4.0,>=2021.1.1 (from torch)
  Downloading https://download.pytorch.org/whl/mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 228.5/228.5 MB 12.3 MB/s eta 0:00:00
Collecting numpy (from torchvision)
  Downloading https://download.pytorch.org/whl/numpy-1.26.3-cp311-cp311-win_amd64.whl (15.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.8/15.8 MB 22.5 MB/s eta 0:00:00
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision)
  Downloading https://download.pytorch.org/whl/pillow-10.2.0-cp311-cp311-win_amd64.whl (2.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 12.0 MB/s eta 0:00:00
Collecting intel-openmp==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch)
  Downloading https://download.pytorch.org/whl/intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 8.0 MB/s eta 0:00:00
Collecting tbb==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch)
  Downloading https://download.pytorch.org/whl/tbb-2021.11.0-py3-none-win_amd64.whl (298 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.3/298.3 kB 9.0 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch)
  Downloading https://download.pytorch.org/whl/MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB)
Collecting mpmath>=0.19 (from sympy->torch)
  Downloading https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 4.8 MB/s eta 0:00:00
Installing collected packages: tbb, mpmath, intel-openmp, typing-extensions, sympy, pillow, numpy, networkx, mkl, MarkupSafe, fsspec, filelock, jinja2, torch, torchvision, torchaudio
Successfully installed MarkupSafe-2.1.5 filelock-3.13.1 fsspec-2024.2.0 intel-openmp-2021.4.0 jinja2-3.1.3 mkl-2021.4.0 mpmath-1.3.0 networkx-3.2.1 numpy-1.26.3 pillow-10.2.0 sympy-1.12 tbb-2021.11.0 torch-2.3.1+cu118 torchaudio-2.3.1+cu118 torchvision-0.18.1+cu118 typing-extensions-4.9.0

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: python.exe -m pip install --upgrade pip
inswapper\CodeFormer\CodeFormer\basicsr\setup.py
inswapper\CodeFormer\CodeFormer\basicsr\train.py
inswapper\CodeFormer\CodeFormer\basicsr\VERSION
inswapper\CodeFormer\CodeFormer\basicsr\version.py
inswapper\CodeFormer\CodeFormer\basicsr\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arcface_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\arch_util.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\codeformer_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\rrdbnet_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vgg_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\vqgan_arch.py
inswapper\CodeFormer\CodeFormer\basicsr\archs\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_sampler.py
inswapper\CodeFormer\CodeFormer\basicsr\data\data_util.py
inswapper\CodeFormer\CodeFormer\basicsr\data\prefetch_dataloader.py
inswapper\CodeFormer\CodeFormer\basicsr\data\transforms.py
inswapper\CodeFormer\CodeFormer\basicsr\data\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\losses.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\loss_util.py
inswapper\CodeFormer\CodeFormer\basicsr\losses\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\metric_util.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\psnr_ssim.py
inswapper\CodeFormer\CodeFormer\basicsr\metrics\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\models\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\deform_conv.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_cuda_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\dcn\src\deform_conv_ext.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\fused_act.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\fused_act\src\fused_bias_act_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\upfirdn2d.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\__init__.py
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d.cpp
inswapper\CodeFormer\CodeFormer\basicsr\ops\upfirdn2d\src\upfirdn2d_kernel.cu
inswapper\CodeFormer\CodeFormer\basicsr\utils\dist_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\download_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\file_client.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\img_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\lmdb_util.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\logger.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\matlab_functions.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\misc.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\options.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\realesrgan_utils.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\registry.py
inswapper\CodeFormer\CodeFormer\basicsr\utils\__init__.py
50 File copiati
inswapper\CodeFormer\CodeFormer\facelib\detection\align_trans.py
inswapper\CodeFormer\CodeFormer\facelib\detection\matlab_cp2tform.py
inswapper\CodeFormer\CodeFormer\facelib\detection\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface.pyinswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_net.pyinswapper\CodeFormer\CodeFormer\facelib\detection\retinaface\retinaface_utils.pyinswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\face_detector.pyinswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\__init__.pyinswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\common.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\experimental.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolo.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5l.yaml
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\yolov5n.yaml
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\models\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\autoanchor.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\datasets.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\extract_ckpt.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\general.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\torch_utils.py
inswapper\CodeFormer\CodeFormer\facelib\detection\yolov5face\utils\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\bisenet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\parsenet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\resnet.py
inswapper\CodeFormer\CodeFormer\facelib\parsing\__init__.py
inswapper\CodeFormer\CodeFormer\facelib\utils\face_restoration_helper.py
inswapper\CodeFormer\CodeFormer\facelib\utils\face_utils.py
inswapper\CodeFormer\CodeFormer\facelib\utils\misc.py
inswapper\CodeFormer\CodeFormer\facelib\utils\__init__.py
28 File copiati



(venv) D:\WM\Fooocus-inswapper>run.bat
[System ARGV] ['launch.py']
Traceback (most recent call last):
  File "D:\WM\Fooocus-inswapper\launch.py", line 22, in <module>
    from modules.launch_util import is_installed, run, python, run_pip, requirements_met, delete_folder_content
  File "D:\WM\Fooocus-inswapper\modules\launch_util.py", line 10, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'

(venv) D:\WM\Fooocus-inswapper>edit run.bat
"edit" non è riconosciuto come comando interno o esterno,
 un programma eseguibile o un file batch.

(venv) D:\WM\Fooocus-inswapper>notepad run.bat

(venv) D:\WM\Fooocus-inswapper>run.bat
[System ARGV] ['launch.py']
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.4.1
[Cleanup] Attempting to delete content of temp dir C:\Users\adolfo\AppData\Local\Temp\fooocus
[Cleanup] Cleanup successful
Downloading: "https://huggingface.co/lllyasviel/misc/resolve/main/xlvaeapp.pth" to D:\WM\Fooocus-inswapper\models\vae_approx\xlvaeapp.pth

100%|██████████████████████████████████████████████████████████████████████████████| 209k/209k [00:00<00:00, 1.94MB/s]
Downloading: "https://huggingface.co/lllyasviel/misc/resolve/main/vaeapp_sd15.pt" to D:\WM\Fooocus-inswapper\models\vae_approx\vaeapp_sd15.pth

100%|██████████████████████████████████████████████████████████████████████████████| 209k/209k [00:00<00:00, 2.43MB/s]
Downloading: "https://huggingface.co/mashb1t/misc/resolve/main/xl-to-v1_interposer-v4.0.safetensors" to D:\WM\Fooocus-inswapper\models\vae_approx\xl-to-v1_interposer-v4.0.safetensors

100%|████████████████████████████████████████████████████████████████████████████| 5.40M/5.40M [00:00<00:00, 13.4MB/s]
Downloading: "https://huggingface.co/lllyasviel/misc/resolve/main/fooocus_expansion.bin" to D:\WM\Fooocus-inswapper\models\prompt_expansion\fooocus_expansion\pytorch_model.bin

100%|██████████████████████████████████████████████████████████████████████████████| 335M/335M [00:10<00:00, 33.2MB/s]
Downloading: "https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/juggernautXL_v8Rundiffusion.safetensors" to D:\WM\Fooocus-inswapper\models\checkpoints\juggernautXL_v8Rundiffusion.safetensors

100%|████████████████████████████████████████████████████████████████████████████| 6.62G/6.62G [02:55<00:00, 40.4MB/s]
Downloading: "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_offset_example-lora_1.0.safetensors" to D:\WM\Fooocus-inswapper\models\loras\sd_xl_offset_example-lora_1.0.safetensors

100%|████████████████████████████████████████████████████████████████████████████| 47.3M/47.3M [00:01<00:00, 29.6MB/s]
Total VRAM 12282 MB, total RAM 65349 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 SUPER : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
Refiner unloaded.
Traceback (most recent call last):
  File "D:\WM\Fooocus-inswapper\launch.py", line 140, in <module>
    from webui import *
  File "D:\WM\Fooocus-inswapper\webui.py", line 17, in <module>
    import modules.pm as pm
  File "D:\WM\Fooocus-inswapper\modules\pm.py", line 7, in <module>
    from diffusers import EulerDiscreteScheduler,  DPMSolverMultistepScheduler, EulerAncestralDiscreteScheduler, DDIMScheduler
ModuleNotFoundError: No module named 'diffusers'

Additional information

No response

"RuntimeError: Found at least two devices"

When using PhotoMaker for the first time:

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Per ChatGPT4:

The error message indicates that there's a device mismatch between the tensors involved in the operation. Specifically, the error states:

vbnet

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

This means that part of your computation is happening on the GPU (cuda:0) and another part on the CPU, which is not allowed during tensor operations in PyTorch. All tensors involved in an operation must be on the same device.

To resolve this issue, ensure that all tensors and models involved in the computation are moved to the same device. You can explicitly move tensors to a device using .to(device) method where device is either "cpu" or a specific CUDA device like "cuda:0".

For example, if you have a tensor tensor and a model model, and you want to perform operations on the GPU, you can do:

python

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
tensor = tensor.to(device)
model = model.to(device)

You should apply this to all tensors and models involved in the operations leading to the error. If some tensors are generated within a function or a library call, check the documentation for those functions or libraries to see if there's a way to specify the device for their internal computations.

In your specific case, it looks like the error is happening within the transformers library during the forward pass of a GPT-2 model. If you're using pre-built functions or classes from a library, ensure that the model and all input tensors are moved to the same device before calling such functions. If the library provides a way to set the device for its internal operations, make sure to use it.

[Feature]Inswapper multi targeting mode: Allow source face image to also be selectable as an index as well as multiple targets per task

Is your feature request related to a problem? Please describe.
This is a double feature ticket.

In Roop face swap in auto1111 there is a feature that allows you to create an array of faces from your source image, and use those to multi target faces in the target image Or just target 1 face (index 0). In the screenshot below you can see that source 0 will swap with target 1, source 2 will swap with target 0 and source 1 will swap with target 2. This is all done in 1 operation

In addition, this helps not having to crop faces out, since you can just target index 1 in an image, and use it for target image index 0.

Feature 1: Multi targeting
Feature 2: source index , although by doing feature 1 feature 2 is done for free.
Describe the idea you'd like

It would be great if we had some sort of parity to this so that we didn't have to do multiple operations. Also this would help not needing to do post processing to a source image if we can just target it in an index as well.

[Bug]: InstantID:AssertionError

Checklist

The issue has not been resolved by following the troubleshooting guide
The issue exists on a clean installation of Fooocus
The issue exists in the current version of Fooocus
The issue has not been reported before recently
The issue has been reported before but has not been fixed yet

What happened?

InstantID: ENABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 568645601431872844
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [['None', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/content/Fooocus-inswapper/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
unload clone 0
[Fooocus Model Management] Moving model(s) has taken 1.65 seconds
[Fooocus] Processing prompts ...
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.11 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (1216, 832)
Preparation time: 1.86 seconds
InstantID: Begin
InstantID: Positive prompt: a photo of a woman
InstantID: Negative prompt:
InstantID: Loading diffusers pipeline into memory.
Traceback (most recent call last):
File "/content/Fooocus-inswapper/modules/async_worker.py", line 976, in worker
handler(task)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/Fooocus-inswapper/modules/async_worker.py", line 873, in handler
imgs = generate_instantid(instantid_image_path, instantid_pose_image_path, instantid_prompt, instantid_negative_prompt, steps, task['task_seed'], width, height, guidance_scale, loras, sampler_name, scheduler_name, async_task, instantid_identitynet_strength_ratio, instantid_adapter_strength_ratio)
File "/content/Fooocus-inswapper/modules/instantid.py", line 144, in generate_instantid
pipe = load_model(loras, sampler_name)
File "/content/Fooocus-inswapper/modules/instantid.py", line 50, in load_model
app = FaceAnalysis(name='antelopev2', root='InstantID', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
File "/usr/local/lib/python3.10/dist-packages/insightface/app/face_analysis.py", line 43, in init
assert 'detection' in self.models
AssertionError
Total time: 1.91 seconds

Steps to reproduce the problem

!pip install pygit2==1.12.2
%cd /content
!git clone https://github.com/machineminded/Fooocus-inswapper.git
!git checkout feature/instantid
%cd /content/Fooocus-inswapper
!chmod +x configure-colab.sh
!/content/Fooocus-inswapper/configure-colab.sh

%pip install -r requirements_versions.txt

!python launch.py --share

enable instantID
nothing happened and in the log i get AssertionError

What should have happened?

create instantID generate image

What browsers do you use to access Fooocus?

Google Chrome

Where are you running Fooocus?

Cloud (Google Colab)

What operating system are you using?

No response

Console logs

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Inswapper: DISABLED
PhotoMaker: DISABLED
InstantID: ENABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 2426708296706601598
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a photo of a woman, full intricate, elegant, highly detailed, wonderful colors, warm dynamic dramatic beautiful new, relaxed, inspired, breathtaking, creative, vibrant, unique, cute, touching, majestic, delicate, symmetry, innocent, pretty, charming, peaceful, iconic, lovely, very, pure, cinematic, artistic, loving, brave, passionate, cheerful, determined
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] a photo of a woman, very beautiful, elegant, cinematic, stunning, highly detailed, real, aesthetic, delicate, dramatic, intricate, rich deep colors, attractive, sublime, coherent, passionate, colorful, surreal, pretty, mystical, open, glowing, determined, extremely, focused, breathtaking, incredible detail, creative, positive light, cute, lovely, infinite, symmetry
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.12 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 3.83 seconds
InstantID: Begin
InstantID: Positive prompt: cinematic still a photo of a woman . emotional, harmonious, vignette, 4k epic detailed, shot on kodak, 35mm photo, sharp focus, high budget, cinemascope, moody, epic, gorgeous, film grain, grainy
InstantID: Negative prompt: (worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, grayscale, bw, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (airbrushed, cartoon, anime, semi-realistic, cgi, render, blender, digital art, manga, amateur:1.3), (3D ,3D Game, 3D Game Scene, 3D Character:1.1), (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)
InstantID: Loading diffusers pipeline into memory.
Traceback (most recent call last):
  File "/content/Fooocus-inswapper/modules/async_worker.py", line 976, in worker
    handler(task)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/Fooocus-inswapper/modules/async_worker.py", line 873, in handler
    imgs = generate_instantid(instantid_image_path, instantid_pose_image_path, instantid_prompt, instantid_negative_prompt, steps, task['task_seed'], width, height, guidance_scale, loras, sampler_name, scheduler_name, async_task, instantid_identitynet_strength_ratio, instantid_adapter_strength_ratio)
  File "/content/Fooocus-inswapper/modules/instantid.py", line 144, in generate_instantid
    pipe = load_model(loras, sampler_name)
  File "/content/Fooocus-inswapper/modules/instantid.py", line 50, in load_model
    app = FaceAnalysis(name='antelopev2', root='InstantID', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
  File "/usr/local/lib/python3.10/dist-packages/insightface/app/face_analysis.py", line 43, in __init__
    assert 'detection' in self.models
AssertionError
Total time: 3.88 seconds
Inswapper: DISABLED
PhotoMaker: DISABLED
InstantID: ENABLED
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 568645601431872844
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [['None', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/content/Fooocus-inswapper/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
unload clone 0
[Fooocus Model Management] Moving model(s) has taken 1.65 seconds
[Fooocus] Processing prompts ...
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.11 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (1216, 832)
Preparation time: 1.86 seconds
InstantID: Begin
InstantID: Positive prompt: a photo of a woman
InstantID: Negative prompt: 
InstantID: Loading diffusers pipeline into memory.
Traceback (most recent call last):
  File "/content/Fooocus-inswapper/modules/async_worker.py", line 976, in worker
    handler(task)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/Fooocus-inswapper/modules/async_worker.py", line 873, in handler
    imgs = generate_instantid(instantid_image_path, instantid_pose_image_path, instantid_prompt, instantid_negative_prompt, steps, task['task_seed'], width, height, guidance_scale, loras, sampler_name, scheduler_name, async_task, instantid_identitynet_strength_ratio, instantid_adapter_strength_ratio)
  File "/content/Fooocus-inswapper/modules/instantid.py", line 144, in generate_instantid
    pipe = load_model(loras, sampler_name)
  File "/content/Fooocus-inswapper/modules/instantid.py", line 50, in load_model
    app = FaceAnalysis(name='antelopev2', root='InstantID', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
  File "/usr/local/lib/python3.10/dist-packages/insightface/app/face_analysis.py", line 43, in __init__
    assert 'detection' in self.models
AssertionError
Total time: 1.91 seconds

Additional information

No response

Kaggle problem

Checklist

The issue has not been resolved by following the troubleshooting guide
The issue exists on a clean installation of Fooocus
The issue exists in the current version of Fooocus
The issue has not been reported before recently
The issue has been reported before but has not been fixed yet

What happened?

Everything works except inswapper. The error is:
File "/opt/conda/lib/python3.10/site-packages/insightface/model_zoo/model_zoo.py", line 91, in get_model
assert osp.exists(model_file), 'model_file %s should exist'%model_file
AssertionError: model_file /kaggle/working/Fooocus-inswapper/inswapper/../inswapper/checkpoints/inswapper_128.onnx should exist

Steps to reproduce the problem

I used this notebook: https://www.kaggle.com/code/aiwalaartist/fooocus-by-aiwalaartist

What should have happened?

Any ideas what could go wrong?

What browsers do you use to access Fooocus?

No response

Where are you running Fooocus?

Cloud (other)

What operating system are you using?

No response

Console logs

File "/opt/conda/lib/python3.10/site-packages/insightface/model_zoo/model_zoo.py", line 91, in get_model
    assert osp.exists(model_file), 'model_file %s should exist'%model_file
AssertionError: model_file /kaggle/working/Fooocus-inswapper/inswapper/../inswapper/checkpoints/inswapper_128.onnx should exist

Additional information

No response

inswapper does not fail gracefully when targeted index in UI is higher than number of faces detected in the image.

Read Troubleshoot

[x] I confirm that I have read the Troubleshoot guide before making this issue.

Describe the problem
When you try to target a face index that is greater than the actual number of faces detected, This is very easy to replicate. If you are swapping an image with x numbers of faces, in " Target Image Index " set it to a number greater than x.

Proposal: This should instead skip and move on to the next task rather than throwing an error and halting.

Full Console Log
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
Source faces: 1
Target faces: 2
Replacing specific face(s) in the target image with specific face(s) from the source image
Traceback (most recent call last):
File "C:\Fooocus inswapper\Fooocus-inswapper\modules\async_worker.py", line 860, in worker
handler(task)
File "C:\Fooocus inswapper\Fooocus-inswapper\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus inswapper\Fooocus-inswapper\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus inswapper\Fooocus-inswapper\modules\async_worker.py", line 812, in handler
imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
File "C:\Fooocus inswapper\Fooocus-inswapper\modules\face_swap.py", line 14, in perform_face_swap
result_image = process([source_image], item, "-1", f"{int(inswapper_target_image_index)}", "../inswapper/checkpoints/inswapper_128.onnx")
File "C:\Fooocus inswapper\Fooocus-inswapper\inswapper\swapper.py", line 179, in process
raise ValueError(f"Target index {target_index} is higher than the number of faces in the target image")
ValueError: Target index 2 is higher than the number of faces in the target image

Not work in google colab.

I am running it in colab, and since the modification that added the photomaker it does not work.
You click on generate and it stays blocked.

Cycle Models

Possibility to set image generations to run multiple models if possible.

[Feature Request]: Replicate support

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits

What would your feature do?

Hi! Great work with this fork.

Putting in a request for adding replicate.com support.

Proposed workflow

It's as simple as adding two files - a cog.yaml that specifies underlying pip packages and other environment details (CUDA version etc)

https://github.com/mrhan1993/Fooocus-API/blob/main/cog.yaml

and a predict.py where the model is loaded and the input types are specified

https://github.com/mrhan1993/Fooocus-API/blob/main/predict.py

You would just have to copy these two files to the repo and then modify to add support for whatever changes you've made to enable InstantID, etc.

Additional information

Further info on the Cog package, which is a wrapper for docker, at https://cog.run

No autosave of inswapper output only initial Fooocus generated image

Describe the problem
The inswapper generated images are not autosaved in the outputs folder only the initial image generated by Fooocus so have to manually save the inswapper version.

This might be intended behaviour but is it possible to save the image only after inswapper has been used or save both images?

how to solve

100%|████████████████████████████████████████████████████████████████| 30/30 [03:52<00:00, 7.75s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 0.29 seconds
Inswapper: Target index: 0.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['AzureExecutionProvider', 'CPUExecutionProvider'], with options: {'AzureExecutionProvider': {}, 'CPUExecutionProvider': {}}
find model: ./checkpoints\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (320, 320)
F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
inswapper-shape: [1, 3, 128, 128]
F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\insightface\utils\transform.py:68: FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
Replacing faces in target image from the left to the right by order
Traceback (most recent call last):
File "F:\New folder (2)\Fooocus-inswapper\modules\async_worker.py", line 859, in worker
handler(task)
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\New folder (2)\Fooocus-inswapper\modules\async_worker.py", line 811, in handler
imgs = perform_face_swap(imgs, inswapper_source_image, inswapper_target_image_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\New folder (2)\Fooocus-inswapper\modules\face_swap.py", line 17, in perform_face_swap
from inswapper.restoration import face_restoration,check_ckpts,set_realesrgan,torch,ARCH_REGISTRY,cv2
File "F:\New folder (2)\Fooocus-inswapper\inswapper\restoration.py", line 10, in
from basicsr.utils import imwrite, img2tensor, tensor2img
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\basicsr_init_.py", line 4, in
from .data import *
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\basicsr\data_init_.py", line 22, in
dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\basicsr\data_init.py", line 22, in
dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ggrov\anaconda3\Lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\basicsr\data\realesrgan_dataset.py", line 11, in
from basicsr.data.degradations import circular_lowpass_kernel, random_mixed_kernels
File "F:\New folder (2)\Fooocus-inswapper\fooocusins_env\Lib\site-packages\basicsr\data\degradations.py", line 8, in
from torchvision.transforms.functional_tensor import rgb_to_grayscale
ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'
Total time: 301.64 seconds

machineminded / fooocus-inswapper Goto Github PK

fooocus-inswapper's Introduction

Fooocus-inswapper

Installation (Windows)

Inswapper Usage

PhotoMaker Usage

PhotoMaker General Usage

PhotoMaker LoRA Usage

Supported PhotoMaker samplers

InstantID Usage

InstantID General Usage

InstantID LoRA Usage

Supported InstantID samplers

Colab

Issues

To Do

fooocus-inswapper's People

Contributors

Stargazers

Watchers

Forkers

fooocus-inswapper's Issues

Is there an existing issue for this?

What would your feature do?

Proposed workflow

Additional information

%pip install -r requirements_versions.txt

Checklist

What happened?

Steps to reproduce the problem

What should have happened?

What browsers do you use to access Fooocus?

Where are you running Fooocus?

What operating system are you using?

Console logs

Additional information

Per ChatGPT4:

Checklist

What happened?

Steps to reproduce the problem

What should have happened?

What browsers do you use to access Fooocus?

Where are you running Fooocus?

What operating system are you using?

Console logs

Additional information

Checklist

What happened?

Steps to reproduce the problem

%pip install -r requirements_versions.txt

What should have happened?

What browsers do you use to access Fooocus?

Where are you running Fooocus?

What operating system are you using?

Console logs

Additional information

Checklist

What happened?

Steps to reproduce the problem

What should have happened?

What browsers do you use to access Fooocus?

Where are you running Fooocus?

What operating system are you using?

Console logs

Additional information

Is there an existing issue for this?

What would your feature do?

Proposed workflow

Additional information

Recommend Projects

Recommend Topics

Recommend Org

Jobs