lgqfhwy / lm-vton

The official pytorch implementation of LM-VTON : "Toward Realistic Virtual Try-on Through Landmark-guided Shape Matching"

LM-VTON (AAAI 2021)

The official pytorch implementation of LM-VTON : "Toward Realistic Virtual Try-on Through Landmark-guided Shape Matching"

Code

1. Download all need data include extra_data.zip, mpv_data.zip, viton_data.zip to the data folder. [Google Drive]

2. Download all checkpoint to the checkpoint folder. [Google Drive]

3. Run in first stage. For example, on mpv dataset, you could refer to first_stage/mpv_scripts/mpv_add_point_vgg_train.sh

4. Run in second stage. For example, on mpv dataset, you could refer to second_stage/content_fusion_mpv_train.sh

License

The use of this code is restricted to non-commercial research and educational purposes.

lm-vton's Issues

Missing Files

The first stage and second stage folders are empty.

Bad results for the second stage of try-on images

Hi @lgqfhwy ，
Thanks for your impressive work. I have tried to reproduce your results on VITON(mentioned as Zalando in your paper) datasets. For the first stage, the warped results of the cloth seem okay. However, there are severe bad results for the second stage.
After I have finished the preparation of the datasets and the environments.
What I did:

• Step1: train the first stage by sh first_stage/viton_scripts/viton_add_point_loss_vgg_add_warp_mask.sh with origin_refined_train_cloth_points.py;
• Step2: test the first stage by sh first_stage/mpv_scripts/mpv_add_point_vgg_train with test datamode and origin_refined_test_cloth_points.py;
• Step3: test the first stage by sh first_stage/mpv_scripts/mpv_add_point_vgg_train with train datamode and origin_refined_test_cloth_points.py, for preparing the training warped cloth for the second stage;
• Step4: train the second stage by sh second_stage/viton_train_scripts/content_fusion_viton_train.sh with content_fusion_mpv_train.py
• Step5: test the second stage by sh second_stage/viton_train_scripts/content_fusion_viton_train.sh with test phase and content_fusion_mpv_test.py ;

I believe I have set all data paths in the correct manner. As a result I got the following try-on results:

Hence these results are quite bad. I want to figure out whether my training operation is correct? Or is there any suggestions for this situation?

I am looking forward to your reply!
Many thanks!

find a bug?

can you release your code or models about the detecting cloth-keypoints ?

transforms.ToTensor () Problems

Hi @lgqfhwy,
Thanks for your impressive work. And I want to re-train the model by myself.

However, when I just follow the environments.yml to set the same settings as you. It shows this kind of error:

img = torch.from_numpy(pic.transpose((2, 0, 1)))
ValueError: axes don't match array

This problem is located in the xxx_dataset.py file when giving a transform to densepose_shape_array to make it a tensor. The same error also to the case of parse_array

Then I follow some suggestions, to change the code in the xxx_dataset.py file :
from:

densepose_shape_tensor = self.transform_one(densepose_shape_array)

to

densepose_shape_tensor = torch.Tensor(densepose_shape_array/255.0)
densepose_shape_tensor = densepose_shape_tensor.unsqueeze(0)

Then the training procedure can start. However, with the increase of training time. There comes another problem in the following (The scale of the generated warped cloth seems strange):

And when I finish the training and give a test with the latest checkpoint, I find the same problem to all warped target cloth:

They are all with mismatched scales and a strange orientation.

Hence, I come here for help, could you kindly share with me some lights on how to tackle these nuts?

Best regards and many thanks,