Problem about training about ladi-vton

[ABaldrati]

Hi @Kangkang625
Thanks for your interest in our work!!

should I first freeze other weights including unet and train textual inversion adapter or should I free other weight and train textual inversion adapter and unet together

First, you should pre-train the inversion adapter, keeping all the other weights (including the unet) frozen.
Then keeping frozen the EMASC and the warping module, you should train the unet and the (pre-trained) inversion adapter together.

I hope this clarify your doubts
Alberto

[snaiws]

I wonder it too.

[Kangkang625]

Thanks for your answer @ABaldrati ！
it's very helpful to my further study，but I still have a little confusion about the unet training.

According to my understanding, the unet should be extended based on the unet of stable diffusion pipeline.
Should I extend the unet, initialize the changed part weight randomly and directly freeze it to pre-train the textual inversion adapter ?

Thanks again for your great work and detailed answer！

[ABaldrati]

According to my understanding, the unet should be extended based on the unet of stable diffusion pipeline.
Should I extend the unet, initialize the changed part weight randomly and directly freeze it to pre-train the textual inversion adapter ?

When we pre-train the inversion adapter we use the standard Stable Diffusion inpainting model. In this phase we do not extend the unet