williamyang1991 / styleganex Goto Github PK
View Code? Open in Web Editor NEW[ICCV 2023] StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces
License: Other
[ICCV 2023] StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces
License: Other
thanks for your awesome works! Can you please let us know when the training code will be released
from argparse import Namespace
from models.psp import pSp
import torch.nn as nn
import torch
import onnx
#Function to Convert to ONNX
def Convert_ONNX():
device = "cuda" if torch.cuda.is_available() else "cpu"
ckpt_path = 'pretrained_models/styleganex_toonify_pixar.pt'
ckpt = torch.load(ckpt_path, map_location='cpu')
opts = ckpt['opts']
opts['checkpoint_path'] = ckpt_path
opts['device'] = device
opts = Namespace(**opts)
torch_model = pSp(opts)
torch_model.cpu()
output_onnx = str("styleganex_toonify_pixar.onnx")
# set the model to inference mode
torch_model.eval()
# The exported model will thus accept inputs of size [batch_size, 1, 224, 224] where batch_size can be variable.
batch_size = 1
# Let's create a dummy input tensor
channel = 3
height = 224
width = 224
torch_input = torch.randn(batch_size, channel, height, width, requires_grad=True)
dynamic_axes= {
'input0': {0: 'batch', 2: 'height', 3: 'width'},
'output0': {0: 'batch', 2: 'height', 3: 'width'}
}
# Export the model
# """
torch.onnx.export(
torch_model, # model being run
torch_input, # model input (or a tuple for multiple inputs)
output_onnx, # where to save the model
export_params=True, # store the trained parameter weights inside the model file
opset_version=15, # the ONNX version to export the model to
# WARNING: DNN inference with torch>=1.12 may require do_constant_folding=False
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input0'], # the model's input names
output_names = ['output0'], # the model's output names
dynamic_axes = dynamic_axes)
# """
print(" ")
print('Model has been converted to ONNX')
# Checks
onnx_model = onnx.load(output_onnx) # load onnx model
onnx.checker.check_model(onnx_model) # check onnx model
print('ONNX export success, saved as %s' % output_onnx)
def main():
Convert_ONNX()
if __name__ == "__main__":
main()
When I run this code,it shows the error below:
torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of convolution for kernel of unknown shape. [Caused by the value '1865 defined in (%1865 : Float(*, *, *, *, strides=[401408, 784, 28, 1], requires_grad=1, device=cpu) = onnx::Reshape[allowzero=0](%1803, %1864), scope: models.psp.pSp::/models.stylegan2.model.Generator::decoder/models.stylegan2.model.StyledConv::conv1/models.stylegan2.model.ModulatedConv2d::conv # /home/yxy/github/StyleGANEX/models/stylegan2/model.py:297:0
)' (type 'Tensor') in the TorchScript graph. The containing node has kind 'onnx::Reshape'.]
I have searched some relative docs,It shows that we can not use dynamic shapes when convert to ONNX, but the doc in pytorch didn`t mention this.
Sorry to bother you again, I met a bug
.
I'm training video toonifiy in a single A100 GPU, as the 1th iter calculate the discriminator loss, this error shows. One thing I noticed is that the GPU memory used reaches 62.2GB/80GB at the peak, and then the error shows up. Can you give me some suggestions?
I installed all req and when I run python app_gradio.py
I get this error:
Traceback (most recent call last):
File "app_gradio.py", line 9, in <module>
from webUI.styleganex_model import Model
File "E:\Ai__Project\StyleGANEX\webUI\styleganex_model.py", line 9, in <module>
import dlib
File "C:\Users\ammar\anaconda3\envs\styleganex\lib\site-packages\dlib\__init__.py", line 19, in <module>
from _dlib_pybind11 import *
ImportError: DLL load failed while importing _dlib_pybind11: The specified module could not be found.
Thanks for your great work! I'm now trying to train a new edit direction: slender. I'm a little confused about the data preparing. In data.config it shows the realign320 for training. but in the paper says training with generated data. besides, is it all the 70000 in-the-wild image are used for training and testing?
I generated output video using video_editing.py, but the output size is different from the input resolution. (The bottom part is cut)
The input is portrait video, and the head is not located at the center. Something like this.
Is there any way to get the same resolution to the original input? it would be great if you can point out the lines needed to be changed...
Always thank you,
Hi fantastic job! I don't understand the output resolution in video editing, it looks like it tracks a single face and zooms in, what would be the best way to return to it's original size in a video editing app could I just do a zoom out x2 and a move x or y or something?
for my example the original size is 1920x1080 and the output is 1920x1632
In the paper, the author wrote as follows:
"In Fig. 2(e), the first-layer feature fails to provide enough spatial information for a valid rotation. In comparison, the 7-th layer has a higher resolution (32 × 32), making it better suited for capturing spatial information."
As long as I know, resolution 32x32 is for 4th layer and resolution 256x256 is for 7th layer.
When running:
python video_editing.py --ckpt STYLEGANEX_MODEL_PATH --data_path FACE_INPUT_PATH
I get:
File "/content/StyleGANEX/video_editing.py", line 81 return ^ SyntaxError: 'return' outside function
What is the function of editing_w?is it train from different age data ?
or styleganex_edit_age.pt is it train from different age data ?
how to learn age change, What is the principle, if want to learn model use my own different age data , What should I do
Getting Unexpected end of JSON input error in google colab when Number of frames to toonify set to 1000. Also, can't toonify more than 3 seconds of length of a video.
Hi. I can't seem to download a video after it has been made on web ui. I get "Download failed - no file" either if I try to save it as html or mp4 file.
Good jobs~ Can u offer the pretrain weight like stylegan2-ffhq-config-f.pt ?
我输出之后resize到原分辨率 发现人物位置变了
Thanks for awesome work!.
How can i obtain a attribute edit vector (exp: smile, glasses, ...)
Thanks for your great work. what's the data organization in video toonify(eg, what's in 'toonify_in': 'data/train/pixar/trainA/'), and other style like arcane, comic, cartoon, ukyio,etc. The toonify dataset in my download was not organized. thank you for your reply
I installed everything right and other scripts are working but gradio doesn't work. I'm on Windows
Traceback (most recent call last):
File "app_gradio.py", line 97, in
main()
File "app_gradio.py", line 75, in main
create_demo_inversion(model.process_inversion, allow_optimization=True)
File "X:\StyleGANEX\webUI\app_task.py", line 313, in create_demo_inversion
api_name='inversion')
File "X:\StyleGANEX\venv\lib\site-packages\gradio\events.py", line 157, in call
trigger_only_on_success=self.trigger_only_on_success,
File "X:\StyleGANEX\venv\lib\site-packages\gradio\blocks.py", line 225, in set_event_trigger
check_function_inputs_match(fn, inputs, inputs_as_dict)
File "X:\StyleGANEX\venv\lib\site-packages\gradio\utils.py", line 749, in check_function_inputs_match
parameter_types = get_type_hints(fn)
File "X:\StyleGANEX\venv\lib\site-packages\gradio\utils.py", line 704, in get_type_hints
return typing.get_type_hints(fn)
File "X:\StyleGANEX\venv\lib\typing.py", line 1013, in get_type_hints
value = _eval_type(value, globalns, localns)
File "X:\StyleGANEX\venv\lib\typing.py", line 263, in _eval_type
return t._evaluate(globalns, localns)
File "X:\StyleGANEX\venv\lib\typing.py", line 467, in _evaluate
eval(self.forward_code, globalns, localns),
File "", line 1, in
NameError: name 'file' is not defined
If it helps some file are redownloaded whenever I try to launch Gradio
100%|█████████████████████████████████████████████████████████████████████████████| 17.5k/17.5k [00:00<00:00, 1.05MB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 4.01M/4.01M [00:02<00:00, 1.58MB/s]
100%|████████████████████████████████████████████████████████████████████████████████| 174k/174k [00:00<00:00, 628kB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 4.11k/4.11k [00:00<00:00, 1.41MB/s]
100%|██████████████████████████████████████████████████████████████████████████████| 42.9k/42.9k [00:00<00:00, 350kB/s]
100%|██████████████████████████████████████████████████████████████████████████████| 74.1k/74.1k [00:00<00:00, 489kB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 1.04M/1.04M [00:00<00:00, 1.26MB/s]
100%|███████████████████████████████████████████████████████████████████████████████| 901k/901k [00:00<00:00, 1.26MB/s]
These are the redownloads
Did you train video face editing (e.g black hair) on aligned data?
According to the paper, it uses a synthetic dataset from Stylegan2 or did I miss something?
(we can simply generate x and y from random latent code w+ with StyleGAN G_0)
Awesome work. Vtoonify was amazing, but StyleGANEX is even better.
Especially, I am interested in face attribute editing (video), but I can only test age and hair editing. As shown in the example (open mouse, smile, gender swap, etc), how can I edit the face attributes?
Is training StyleGANEX necessary?
It would be great if you could elaborate how to get different face attribute editing step by step.
Thank you! :)
Can StyleGANEX be used for Face Restoration tasks without zooming in,like https://github.com/yangxy/GPEN
I tried to apply my styles found through StyleCLIP with shape [18,512]
to codes
variable in psp forward function, but they don't seem to work in hair/age or inversion (after optimization) networks. Even though generator is standard Stylegan. Seems like first_layer_feats
from encoder suppress my StyleCLIP edit. But I see that random styles obtained through mapping network from 512 random vectors work in your example. Can I use StyleCLIP or somehow obtain my own styles?
hi @williamyang1991
why the padding is calculated as self.padding = kernel_size // 2 + dilation - 1
instead of self.padding = (kernel_size - 1) // 2 * dilation
Dear Williamyang1991:
thanks for your nice work.
I want to generate the video with younger face, is the following command correct?
python video_editing.py --ckpt pretrained_models\styleganex_edit_age.pt --data_path data\2.mp4 --scale_factor 2
what's the real meaning of scale_factor?
thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.