ygtxr1997 / celebbasis Goto Github PK
View Code? Open in Web Editor NEWOfficial Implementation of 'Inserting Anybody in Diffusion Models via Celeb Basis'
Home Page: https://celeb-basis.github.io/
License: MIT License
Official Implementation of 'Inserting Anybody in Diffusion Models via Celeb Basis'
Home Page: https://celeb-basis.github.io/
License: MIT License
I got this error:
Epoch 0: 80%|▊| 800/1001 [06:35<01:39, 2.03it/s, loss=0.0671, v_num=0, train/loss_simple_step=0.0197, train/loss_vlb_step=7.03e-5, train/loss_step=0Epoch 0, global step 799: val/loss_simple_ema was not in top 1
Average Epoch time: 395.59 seconds
Average Peak memory 19447.18MiB
Epoch 0: 80%|▊| 801/1001 [06:35<01:38, 2.02it/s, loss=0.0671, v_num=0, train/loss_simple_step=0.0197, train/loss_vlb_step=7.03e-5, train/loss_step=0
Saving latest checkpoint...
Traceback (most recent call last):
File "main_id_embed.py", line 817, in
trainer.test(model, data)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 954, in _test_impl
results = self._run(model, ckpt_path=self.tested_ckpt_path)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1128, in _run
verify_loop_configurations(self)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 42, in verify_loop_configurations
__verify_eval_loop_configuration(trainer, model, "test")
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 186, in __verify_eval_loop_configuration
raise MisconfigurationException(f"No {loader_name}()
method defined to run Trainer.{trainer_method}
.")
pytorch_lightning.utilities.exceptions.MisconfigurationException: No test_dataloader()
method defined to run Trainer.test
.
env:
I have configured the environment by following these steps:
I used the command conda env create -f environment.yaml to create the environment based on the specifications provided in the environment.yaml file.
Then, I activated the environment using the command conda activate celebbasis. By changing the environment name to "celebbasis," I ensured that I am working within this specific environment.
Regarding the dependencies mentioned:
The line # - -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers refers to the "taming-transformers" library. Due to network issues, I independently installed this dependency using the command pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers.
Similarly, the line # - -e git+https://github.com/openai/CLIP.git@main#egg=clip refers to the "CLIP" library. You installed this dependency separately as well, using the command pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip.
Next ,
About Face Alignment
I align the images in Img, like id0(1).jpg,...,id0(10).jpg. I think this is a minor issue.
Then, I run bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt and meet the top error
Thanks
Thanks for the really cool work, excited to try it.
Any timeline for code release?
Feel fee to close if you feel not relevant.
Do you have a google colab for this?
I'm very confused, I see there are a lot of images in the /infer_images folder, including /reg or regularization images, and the stylegan3 examples under /ffhq.
However, the pickle file I made in step 00 (face alignment) doesn't generate or include reg images, so when it's unpickled it just contains my (single) aligned cropped photo (I used the pretty print package to display the unpickled dictionary)
and gives me an error message about missing regularization images.
the pickle_path seems to default to another hardcoded location from a fellow named gavin, and this path includes stylegan3 in the name, hence my confusion
Please advise!
thank you for the amazing work you provide. I see that ' finetuning with multiple persons' on your TODO list .
when do you schedule to add it to your github repository?
@vinthony @ygtxr1997
on line 6 of /ldm/models/autoencoder.py
there is an import for a module that doesn't exist in this repo
I had to git clone this repo https://github.com/CompVis/taming-transformers/tree/master
and copy over the /taming folder to the root directory so autoencoder.py
would run successfully
From my experience - does an amazing job.
Getting this error:
(celebbasis) user@amax:/data_heat/rjt_project/CelebBasis$ bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt
Global seed set to 23
Running on GPUs 0,
Loading model from ./weights/sd-v1-4-full-ema.ckpt
Traceback (most recent call last):
File "main_id_embed.py", line 621, in
model = load_model_from_config(config, opt.actual_resume)
File "main_id_embed.py", line 32, in load_model_from_config
pl_sd = torch.load(ckpt, map_location="cpu")
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/torch/serialization.py", line 600, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/torch/serialization.py", line 242, in init
super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main_id_embed.py", line 833, in
if trainer.global_rank == 0:
NameError: name 'trainer' is not defined
env:
I have configured the environment by following these steps:
I used the command conda env create -f environment.yaml to create the environment based on the specifications provided in the environment.yaml file.
Then, I activated the environment using the command conda activate celebbasis. By changing the environment name to "celebbasis," I ensured that I am working within this specific environment.
Regarding the dependencies mentioned:
The line # - -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers refers to the "taming-transformers" library. Due to network issues, I independently installed this dependency using the command pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers.
Similarly, the line # - -e git+https://github.com/openai/CLIP.git@main#egg=clip refers to the "CLIP" library. You installed this dependency separately as well, using the command pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip.
Next ,
About Face Alignment
I align the images in Img, like id0(1).jpg,...,id0(10).jpg. I think this is a minor issue.
Then, I run bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt and meet the top error
Thanks
Getting this error:
│ /home/xander/Projects/cog/GitHub_repos/CelebBasis/ldm/models/diffusion/ddpm.py:1087 in p_losses │
│ │
│ 1084 │ │ loss_simple = self.get_loss(model_output, target, mean=False).mean([1, 2, 3]) │
│ 1085 │ │ loss_dict.update({f'{prefix}/loss_simple': loss_simple.mean()}) │
│ 1086 │ │ │
│ ❱ 1087 │ │ logvar_t = self.logvar[t].to(self.device) │
│ 1088 │ │ loss = loss_simple / torch.exp(logvar_t) + logvar_t │
│ 1089 │ │ # loss = loss_simple / torch.exp(self.logvar) + self.logvar │
│ 1090 │ │ if self.learn_logvar: │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)
Changing
logvar_t = self.logvar[t].to(self.device)
to
logvar_t = self.logvar[t.cpu()].to(self.device)
Solved this, but that feels like the wrong place to fix this bug...
sd-v1-4-full-ema.ckpt could not be found in the provided link
Very nice work and code repository!
I wonder if you can provide any source images and prompt for evaluation.
Besides, clear instruction for how to evaluate the customized models on the metrics of "Prompt" and "Detect" would be appreciated!
Thank you!
Thanks for this work.
What changes would be required to use SDXL instead of the 1.4 checkpoint?
Where is the list of the 1.500 celebs?
I needed to make a change in /evaluation/face_align/PIPNet/start_align.py
on line 40 during the DataLoader object instance from num_workers=1
to num_workers=0
, or else the tqdm was just hanging indefinitely, as seen in this forum post talking about num_workers>0 https://discuss.pytorch.org/t/dataloader-slower-with-num-workers-0/100970 (up to 50 times slower)
Perhaps it depends on system configuration, but this is what I needed to do to get it to run instantly, and not hang indefinitely, hope it helps others out!
I've been running it so far via Windows Subsystem for Linux 2, and the 01_start_train.sh bash script was crashing at the default 16 GB of RAM for my 32 GB Ubuntu install. I had to increase it to 26 GB (Windows 11 needs about 4 GB to operate) via the .wslconfig or it was getting killed due to lack of memory https://fizzylogic.nl/2023/01/05/how-to-configure-memory-limits-in-wsl2
I think there is an option to use the gpu to load the models directly, rather than copy them into RAM first, but I suppose this wouldn't be good for low VRAM GPUs (I have a 3090).
I get the following error running bash 00_align_face.sh <...>/ori <...>/ffhq
(sd) wsl@MJDFW:~/repos/CelebBasis$ bash 00_align_face.sh /home/wsl/repos/CelebBasis/my_imgs/ori/ /home/wsl/repos/CelebBasis/my_imgs/ffhq/
running build_ext
skipping 'nms/cpu_nms.c' Cython extension (up-to-date)
Traceback (most recent call last):
File "./evaluation/face_align/PIPNet/start_align.py", line 85, in <module>
main(opt)
File "./evaluation/face_align/PIPNet/start_align.py", line 35, in main
folder_dataset = FolderAlignCrop(
File "/home/wsl/repos/CelebBasis/evaluation/face_align/PIPNet/alignment/dataloader.py", line 40, in __init__
self.net, self.detector = get_lmk_model()
File "/home/wsl/repos/CelebBasis/evaluation/face_align/PIPNet/lib/tools.py", line 62, in get_lmk_model
state_dict = torch.load(weight_file, map_location=device)
File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 594, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 211, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/gavin/code/FaceSwapping/inference/PIPNet/weights/epoch59.pth'
due to the hardcoded paths in the get_lmk_model() function in evaluation/face_align/PIPNet/lib/tools.py
as seen here
As we can see here, there should be PIPNet weights (and epoch59.pth ?) which need to be downloaded (and instructions for doing so in the README.md)
Please advise, thanks!
using the 03_extract.sh to extract the embeddings, then fill the tensor of the embedding files into a webui style embedding file, and then good to go.
And from experiments, it seems 799 steps usually lead to a better result than that from fewer or more steps.
I found in the face_id.py file, there are classes that could specifically use the celeb-asian dataset, but the config files do not have any record of using those classes. Could I simply swap the default classes for those?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.