williamyang1991 / styleganex Goto Github PK

Firstly thank you for the wonderful project.

But I am stuck when running it to Age editing, I can't find a param to change the age
Can you explain which i param will make an object in the image older than
Thank you

thanks for your awesome works! Can you please let us know when the training code will be released

from argparse import Namespace
from models.psp import pSp
import torch.nn as nn
import torch
import onnx

#Function to Convert to ONNX 
def Convert_ONNX(): 
    device = "cuda" if torch.cuda.is_available() else "cpu"
    ckpt_path = 'pretrained_models/styleganex_toonify_pixar.pt'
    ckpt = torch.load(ckpt_path, map_location='cpu')
    opts = ckpt['opts']
    opts['checkpoint_path'] = ckpt_path
    opts['device'] =  device
    opts = Namespace(**opts)
    torch_model = pSp(opts)
    torch_model.cpu()

    output_onnx = str("styleganex_toonify_pixar.onnx")

    # set the model to inference mode 
    torch_model.eval() 

    # The exported model will thus accept inputs of size [batch_size, 1, 224, 224] where batch_size can be variable.
    batch_size = 1 
    # Let's create a dummy input tensor
    channel = 3
    height = 224
    width = 224
    torch_input = torch.randn(batch_size, channel, height, width, requires_grad=True)

    dynamic_axes= {
        'input0': {0: 'batch', 2: 'height', 3: 'width'},
        'output0': {0: 'batch', 2: 'height', 3: 'width'}
    }

    # Export the model
    # """ 
    torch.onnx.export(
         torch_model,         # model being run 
         torch_input,       # model input (or a tuple for multiple inputs) 
         output_onnx,       # where to save the model  
         export_params=True,  # store the trained parameter weights inside the model file 
         opset_version=15,    # the ONNX version to export the model to 
         # WARNING: DNN inference with torch>=1.12 may require do_constant_folding=False
         do_constant_folding=True,  # whether to execute constant folding for optimization
         input_names = ['input0'],   # the model's input names 
         output_names = ['output0'], # the model's output names 
         dynamic_axes = dynamic_axes)
    # """
    
    print(" ") 
    print('Model has been converted to ONNX')

    # Checks
    onnx_model = onnx.load(output_onnx)  # load onnx model
    onnx.checker.check_model(onnx_model)  # check onnx model

    print('ONNX export success, saved as %s' % output_onnx)


def main():
    Convert_ONNX()

if __name__ == "__main__":
    main()

When I run this code,it shows the error below:

torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of convolution for kernel of unknown shape. [Caused by the value '1865 defined in (%1865 : Float(*, *, *, *, strides=[401408, 784, 28, 1], requires_grad=1, device=cpu) = onnx::Reshape[allowzero=0](%1803, %1864), scope: models.psp.pSp::/models.stylegan2.model.Generator::decoder/models.stylegan2.model.StyledConv::conv1/models.stylegan2.model.ModulatedConv2d::conv # /home/yxy/github/StyleGANEX/models/stylegan2/model.py:297:0
)' (type 'Tensor') in the TorchScript graph. The containing node has kind 'onnx::Reshape'.]

I have searched some relative docs,It shows that we can not use dynamic shapes when convert to ONNX, but the doc in pytorch didn`t mention this.

Sorry to bother you again, I met a bug
.
I'm training video toonifiy in a single A100 GPU, as the 1th iter calculate the discriminator loss, this error shows. One thing I noticed is that the GPU memory used reaches 62.2GB/80GB at the peak, and then the error shows up. Can you give me some suggestions?

Great job! I have some questions about the code. If I edit images, do I need to do the following? Will removal have an impact on the results?

I installed all req and when I run python app_gradio.py

I get this error:

Traceback (most recent call last):
  File "app_gradio.py", line 9, in <module>
    from webUI.styleganex_model import Model
  File "E:\Ai__Project\StyleGANEX\webUI\styleganex_model.py", line 9, in <module>
    import dlib
  File "C:\Users\ammar\anaconda3\envs\styleganex\lib\site-packages\dlib\__init__.py", line 19, in <module>
    from _dlib_pybind11 import *
ImportError: DLL load failed while importing _dlib_pybind11: The specified module could not be found.

Thanks for your great work! I'm now trying to train a new edit direction: slender. I'm a little confused about the data preparing. In data.config it shows the realign320 for training. but in the paper says training with generated data. besides, is it all the 70000 in-the-wild image are used for training and testing?

This is awesome job. But I have some problem, when using huggingface "Image Face Editing" strange image are generated. The texture looks like "leather", only light hair color 1.3. Below is the output image

The Parameter:

I generated output video using video_editing.py, but the output size is different from the input resolution. (The bottom part is cut)
The input is portrait video, and the head is not located at the center. Something like this.

Is there any way to get the same resolution to the original input? it would be great if you can point out the lines needed to be changed...

Always thank you,

Hi fantastic job! I don't understand the output resolution in video editing, it looks like it tracks a single face and zooms in, what would be the best way to return to it's original size in a video editing app could I just do a zoom out x2 and a move x or y or something?

for my example the original size is 1920x1080 and the output is 1920x1632

In the paper, the author wrote as follows:
"In Fig. 2(e), the first-layer feature fails to provide enough spatial information for a valid rotation. In comparison, the 7-th layer has a higher resolution (32 × 32), making it better suited for capturing spatial information."
As long as I know, resolution 32x32 is for 4th layer and resolution 256x256 is for 7th layer.

When running:

python video_editing.py --ckpt STYLEGANEX_MODEL_PATH --data_path FACE_INPUT_PATH

I get:

File "/content/StyleGANEX/video_editing.py", line 81 return ^ SyntaxError: 'return' outside function

What is the function of editing_w？is it train from different age data ？

or styleganex_edit_age.pt is it train from different age data ？

how to learn age change， What is the principle, if want to learn model use my own different age data , What should I do

Getting Unexpected end of JSON input error in google colab when Number of frames to toonify set to 1000. Also, can't toonify more than 3 seconds of length of a video.

Hi. I can't seem to download a video after it has been made on web ui. I get "Download failed - no file" either if I try to save it as html or mp4 file.

Good jobs~ Can u offer the pretrain weight like stylegan2-ffhq-config-f.pt ？

我输出之后resize到原分辨率 发现人物位置变了

Thanks for awesome work!.
How can i obtain a attribute edit vector (exp: smile, glasses, ...)

Thanks for your great work. what's the data organization in video toonify(eg, what's in 'toonify_in': 'data/train/pixar/trainA/'), and other style like arcane, comic, cartoon, ukyio,etc. The toonify dataset in my download was not organized. thank you for your reply

I installed everything right and other scripts are working but gradio doesn't work. I'm on Windows
Traceback (most recent call last):
File "app_gradio.py", line 97, in
main()
File "app_gradio.py", line 75, in main
create_demo_inversion(model.process_inversion, allow_optimization=True)
File "X:\StyleGANEX\webUI\app_task.py", line 313, in create_demo_inversion
api_name='inversion')
File "X:\StyleGANEX\venv\lib\site-packages\gradio\events.py", line 157, in call
trigger_only_on_success=self.trigger_only_on_success,
File "X:\StyleGANEX\venv\lib\site-packages\gradio\blocks.py", line 225, in set_event_trigger
check_function_inputs_match(fn, inputs, inputs_as_dict)
File "X:\StyleGANEX\venv\lib\site-packages\gradio\utils.py", line 749, in check_function_inputs_match
parameter_types = get_type_hints(fn)
File "X:\StyleGANEX\venv\lib\site-packages\gradio\utils.py", line 704, in get_type_hints
return typing.get_type_hints(fn)
File "X:\StyleGANEX\venv\lib\typing.py", line 1013, in get_type_hints
value = _eval_type(value, globalns, localns)
File "X:\StyleGANEX\venv\lib\typing.py", line 263, in _eval_type
return t._evaluate(globalns, localns)
File "X:\StyleGANEX\venv\lib\typing.py", line 467, in _evaluate
eval(self.forward_code, globalns, localns),
File "", line 1, in
NameError: name 'file' is not defined

If it helps some file are redownloaded whenever I try to launch Gradio
100%|█████████████████████████████████████████████████████████████████████████████| 17.5k/17.5k [00:00<00:00, 1.05MB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 4.01M/4.01M [00:02<00:00, 1.58MB/s]
100%|████████████████████████████████████████████████████████████████████████████████| 174k/174k [00:00<00:00, 628kB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 4.11k/4.11k [00:00<00:00, 1.41MB/s]
100%|██████████████████████████████████████████████████████████████████████████████| 42.9k/42.9k [00:00<00:00, 350kB/s]
100%|██████████████████████████████████████████████████████████████████████████████| 74.1k/74.1k [00:00<00:00, 489kB/s]
100%|█████████████████████████████████████████████████████████████████████████████| 1.04M/1.04M [00:00<00:00, 1.26MB/s]
100%|███████████████████████████████████████████████████████████████████████████████| 901k/901k [00:00<00:00, 1.26MB/s]

These are the redownloads

Did you train video face editing (e.g black hair) on aligned data?
According to the paper, it uses a synthetic dataset from Stylegan2 or did I miss something?
(we can simply generate x and y from random latent code w+ with StyleGAN G_0)

Awesome work. Vtoonify was amazing, but StyleGANEX is even better.

Especially, I am interested in face attribute editing (video), but I can only test age and hair editing. As shown in the example (open mouse, smile, gender swap, etc), how can I edit the face attributes?

Is training StyleGANEX necessary?
It would be great if you could elaborate how to get different face attribute editing step by step.

Thank you! :)

Can StyleGANEX be used for Face Restoration tasks without zooming in，like https://github.com/yangxy/GPEN

I tried to apply my styles found through StyleCLIP with shape [18,512] to codes variable in psp forward function, but they don't seem to work in hair/age or inversion (after optimization) networks. Even though generator is standard Stylegan. Seems like first_layer_feats from encoder suppress my StyleCLIP edit. But I see that random styles obtained through mapping network from 512 random vectors work in your example. Can I use StyleCLIP or somehow obtain my own styles?

Hi, I'm interested in your work, Here are my inversion results, the inversion is a bit fuzzy (blocky pixels), Is this normal?

hi @williamyang1991
why the padding is calculated as self.padding = kernel_size // 2 + dilation - 1 instead of self.padding = (kernel_size - 1) // 2 * dilation

Dear Williamyang1991:

thanks for your nice work.

I want to generate the video with younger face, is the following command correct?

python video_editing.py --ckpt pretrained_models\styleganex_edit_age.pt --data_path data\2.mp4 --scale_factor 2

what's the real meaning of scale_factor?

thanks.