open-mmlab / mmfashion

Open-source toolbox for visual fashion analysis based on PyTorch

MMFashion

Introduction

[Technical Report]

MMFashion is an open source visual fashion analysis toolbox based on PyTorch. It is a part of the open-mmlab project developed by Multimedia Lab, CUHK.

Updates

[2019-11-01] MMFashion v0.1 is released.

[2020-02-14] MMFashion v0.2 is released, adding consumer-to-shop retrieval module.

[2020-04-27] MMFashion v0.3 is released, adding fashion segmentation and parsing module.

[2020-05-04] MMFashion v0.4 is released, adding fashion compatibility and recommendation module.

[2020-12-08] MMFashion v0.5 is released, adding virtual try-on module.

Features

• Flexible: modular design and easy to extend

• Friendly: off-the-shelf models for layman users

• Comprehensive: support a wide spectrum of fashion analysis tasks

  • Fashion Attribute Prediction
  • Fashion Recognition and Retrieval
  • Fashion Landmark Detection
  • Fashion Parsing and Segmentation
  • Fashion Compatibility and Recommendation
  • Fashion Virtual Try-On

Requirements

• Python 3.5+
• PyTorch 1.0.0+
• mmcv

Installation

git clone --recursive https://github.com/open-mmlab/mmfashion.git
cd mmfashion
python setup.py install

Another option: Docker Image

We provide a Dockerfile to build an image.

# build an image with PyTorch 1.5, CUDA 10.1
docker build -t mmfashion docker/

Run it with

docker run --gpus all --shm-size=8g -it mmfashion

Get Started

Please refer to GETTING_STARTED.md for the basic usage of MMFashion.

Data Preparation

Please refer to DATA_PREPARATION.md for the dataset specifics of MMFashion.

Model Zoo

Please refer to MODEL_ZOO.md for a comprehensive set of pre-trained models in MMFashion.

Contributing

We appreciate all contributions to improve MMFashion. Please refer to CONTRIBUTING.md for the contributing guideline.

Related Tools

• fashion-detection
• fashion-landmarks
• fashion-cut

License

This project is released under the Apache 2.0 license.

Team

• Xin Liu (veralauee)
• Jiancheng Li (lijiancheng0614)
• Jiaqi Wang (myownskyW7)
• Ziwei Liu (liuziwei7)

Citation

@inproceedings{mmfashion,
  title={MMFashion: An Open-Source Toolbox for Visual Fashion Analysis},
  author={Liu, Xin and Li, Jiancheng and Wang, Jiaqi and Liu, Ziwei},
  booktitle={{ACM Multimedia 2021, Open Source Software Competition},
  year={2021}
}

mmfashion's Issues

Installation failed?

Tried,

$ conda create --name mm python=3.5 ipython jupyter
$ conda activate mm
$ pip install mmfashion

Collecting mmfashion
Could not find a version that satisfies the requirement mmfashion (from versions: )
No matching distribution found for mmfashion

Also tried,

git clone https://github.com/open-mmlab/mmfashion.git
python setup.py install

This installed a directory ~/anaconda3/envs/ds/lib/python3.6/site-packages/mmcv-0.2.14-py3.6-macosx-10.7-x86_64.egg/ with EGG-INFO/

And leads to error when import:

In [1]: from mmfashion.apis import (init_dist, get_root_logger, test_predictor)                                                                                    
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-5635c4f0c829> in <module>
----> 1 from mmfashion.apis import (init_dist, get_root_logger, test_predictor)

ModuleNotFoundError: No module named 'mmfashion.apis'

[Bug] Error when specifying testing gpu_id

When I try to run multiple evaluations I would want them to run on dfferent gpus, to this end, I changed the respective configuration files,

gpus = dict(train=[0, 1, 2, 3], test=[1])

However I encountered the following error:

load checkpoint from: checkpoints/retrieval_clothes_resnet50/epoch_150.pth                            │+-----------------------------------------------------------------------------+
Traceback (most recent call last):                                                                    │| Processes:                                                       GPU Memory |
  File "tools/test_retriever.py", line 88, in <module>                                                │|  GPU       PID   Type   Process name                             Usage      |
    main()                                                                                            │|=============================================================================|
  File "tools/test_retriever.py", line 84, in main                                                    │|    0      1327      G   /usr/lib/xorg/Xorg                            18MiB |
    logger=logger)                                                                                    │|    0      1360      G   /usr/bin/gnome-shell                          49MiB |
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3│|    0      1541      G   /usr/lib/xorg/Xorg                           190MiB |
.7.egg/mmfashion/apis/test_retriever.py", line 40, in test_retriever                                  │|    0      1670      G   /usr/bin/gnome-shell                         262MiB |
    _non_dist_test(model, query_set, gallery_set, cfg, validate=validate)                             │|    0     12106      C   python                                      2871MiB |
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3│+-----------------------------------------------------------------------------+
.7.egg/mmfashion/apis/test_retriever.py", line 74, in _non_dist_test                                  │                                                                                                     
    query_embeds = _process_embeds(query_set, model, cfg)                                             │                                                                                                     
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3│                                                                                                     
.7.egg/mmfashion/apis/test_retriever.py", line 61, in _process_embeds                                 │                                                                                                     
    embed = model(data['img'], data['landmark'], return_loss=False)                                   │                                                                                                     
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/torch/nn/modules/module.py"│                                                                                                     
, line 541, in __call__                                                                               │                                                                                                     
    result = self.forward(*input, **kwargs)                                                           │                                                                                                     
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/torch/nn/parallel/data_para│                                                                                                     
llel.py", line 146, in forward                                                                        │                                                                                                     
    "them on device: {}".format(self.src_device_obj, t.device))                                       │                                                                                                     
RuntimeError: module must have its parameters and buffers on device cuda:1 (device_ids[0]) but found o│                                                                                                     
ne of them on device: cuda:0             

It seems that there's something in the core module that by default load tensors into gpu 0, maybe some thing like x = x.cuda(0)? Hope this problem can be fixed later. Thanks!

Error about 'idx2id'

python tools/test_retriever.py --config configs/retriever_in_shop/roi_retriever_vgg.py --checkpoint checkpoint/Retrieve_in_shop/vgg/epoch_100.pth

When I execute it like this I got the following error

init() got an unexpected keyword argument 'idx2id'

When I try to find the reference from configs/retriever_in_sjop/roi_retriever_vgg.py
I find the following definition in query and gallery dict.
idx2id=os.path.join(data_root, 'Anno/query_idx2id.txt')

But after I execute /data/prepare_in_shop.py I find there does not have a file named query_idx2id.txt or gallery_idx2id.txt
Is this error caused by the data?

looking forward to your reply

Can't load fashion segmentation model

Hi! Could you help me with my issue?
I have downloaded epoch_15.pth for fashion segmentation from Model_Zoo, but init_detector returns "The model and loaded state dict do not match exactly" for all mask_rsnn configs. Where can I get a suitable one?

Wrong predictions

Hi,

I followed the process of setup, downloaded checkpoints for vgg16.pth, latest.pth for global attr.

Anytime I run the demo I get the same output no matter the image.

[ Top3 Prediction ]
print
lace
knit
[ Top5 Prediction ]
print
lace
knit
sleeve
maxi
[ Top10 Prediction ]
print
lace
knit
sleeve
maxi
shirt
denim
chiffon
floral
striped

Looking for a guide on what I could be doing wrong?

Appreciate it and thanks for you work.

Accuracy about in-shop retriever

As with the latest code by the commit of aea9b23
I execute the code by

python demo/demo/test_retriever.py 

the parse in test_retriever.py are the following

parser = argparse.ArgumentParser(
        description='MMFashion In-shop Clothes Retriever Demo')
    parser.add_argument(
        '--input',
        type=str,
        help='input image path',
        default='demo/retrieve_demo1.jpg')
    parser.add_argument(
        '--config',
        help='train config file path',
        default='configs/retriever_in_shop/global_retriever_vgg_loss_id.py')
    parser.add_argument(
        '--checkpoint',
        type=str,
        default='checkpoint/Retrieve/vgg/global/epoch_100.pth',
        help='the checkpoint file to resume from')
    parser.add_argument(
        '--use_cuda', type=bool, default=True, help='use gpu or not')
    args = parser.parse_args()
    return args

checkpoint/Retrieve/vgg/global/epoch_100.pth is downloaded from following

Backbone | Pooling | Loss | Top-5 Acc. | Download
VGG-16 | Global Pooling | Cross-Entropy | 38.76 | model

I got the following result:

pretrained model None
Retrived Top5 Results
Retrieved Image img/WOMEN/Jackets_Coats/id_00002609/02_3_back.jpg
Retrieved Image img/WOMEN/Tees_Tanks/id_00004096/06_1_front.jpg
Retrieved Image img/WOMEN/Jackets_Coats/id_00006337/02_4_full.jpg
Retrieved Image img/WOMEN/Shorts/id_00003973/06_2_side.jpg
Retrieved Image img/WOMEN/Shorts/id_00006962/01_2_side.jpg

The result still seems to be not correspond with the demo image

I really want to present the same result as with the demo you show here
https://github.com/open-mmlab/mmfashion/blob/master/misc/demo_retrieval.gif

Can you give me some hint? Thank you

wrong code implementation?

    normalized_error, det_percent = self.compute_distance(
        vis * pred_lm_np, vis * landmark_np)

shouldn't it be

    normalized_error, det_percent = self.compute_distance(
        pred_vis * pred_lm_np, vis * landmark_np)

then the test result in modelzoo are wrong projection?

The model and loaded state dict do not match exactly

Dear Authors,

When I try the landmark detector demo, I meet the following issues. Please give one good demo or example. Thanks

pretrained model checkpoint/vgg16.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: classifier.0.weight, classifier.0.bias, classifier.3.weight, classifier.3.bias, classifier.6.weight, classifier.6.bias

model built
load checkpoint from: checkpoint/LandmarkDetect/global/landmark_detect_best.pth
F:\Python\Anaconda3\envs\cuhk_mmlab\lib\site-packages\torch\nn\functional.py:1569: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
detected landmark [ 49.13241 134.01773] [169.72472 133.90952]
detected landmark [169.72472 133.90952] [ 67.13259 172.5399 ]
detected landmark [ 67.13259 172.5399 ] [151.30598 170.36455]

qs about img dataset

Thank you for the awesome dataset and codes.

In "img" directory of your dataset,
what is the difference between img/CLOTHING/pants vs img/TROUSERS/pants ??
I see the id numbers are different inside each directory, but why do we need two separate "pants" category??

about the retrieve

hello! i've met a problem ''NoneType' object has no attribute 'unsqueeze'' when i try the roi pooling model, the config file is '/roi_retriever_vgg_loss_id.py', i guess i should change line 'query_embed = model(img_tensor, landmark=None, return_loss=False)' in demo/test_retriever.py, but i don't know the input of landmark, could you please tell me? besides, if i want change the dataset to my own, when retrieving, which files should i generate before(i guess not all in data/In-shop need prepared)?

the accuracy of In-Shop Clothes Retrieval model

Hello
Thank you for your great work!
When I run python3 demo/test_retriever.py and load the model using:

--config configs/retriever/roi_retriever_vgg.py
--input demo/retrieve_demo1.jpg
--checkpoint checkpoint/Retrieve/vgg/epoch_190.pth

And got:

pretrained model checkpoint/vgg16.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: classifier.0.weight, classifier.0.bias, classifier.3.weight, classifier.3.bias, classifier.6.weight, classifier.6.bias

Retrived top5 results are very inaccurate, no picture is right. Is there something wrong with my way of using it? If so, how to choose args?I also want to know how to get the matching degree of each picture？
The gallery_embeds time is so long ，It's about four minutes.
Forgive me for being stupid.Could you please give me some suggestions.
Thank you very much

Where to download backbone for Resnet models

Hi,

I was wondering if you also provided backbone files for Resnet models. I could only find one for Vgg at Model_Zoo.md. Would greatly appreciate some help in this area!

Thanks!

How to extract detailed features for a clothes?

Hi,
I was able to run the following code to obtain top 10 features of a clothes:
python demo/test_predictor.py --config configs/attribute_predict/global_predictor_resnet_attr.py --checkpoint checkpoint/resnet/model_best_resnet_global.pth --input demo/attr_pred_demo2.jpg

Is there a way to extract more detailed features of an image in (e.g., a list of about 100 or 200 values)? I want to use these features to further check similarity of two clothes.

Thanks,
Kaida

How to predict single img's attribute?

Thanks for your great work!
And I have a question here, how to predict my own imgs' attribute? Is there a simple api to predict one single img's attribute/box/landmark？

Travis CI is not enabled for this repo

Other repos are enabled at
https://travis-ci.com/open-mmlab
https://travis-ci.org/open-mmlab
But this repo is not so there is no automated testing.

question about the paper DeepFashion(In-Shop Clothes Retrieval)

Hi,
In the paper section 4.1 result -> In-Shop Clothes Retrieval, did you calculate the retrieval results from all the categories or only from the same category?
For example, if the query is an img inside women-> skirts folder, did you only choose the potential match from women-> skirts folder and then calculate top k accuracy, or did you choose from all the imgs inside in-shop folder and then measure the top k accuracy?

Thanks in advance

Some questions about training Consumer_to_shop

Reading attributes feels like there's something wrong.

mmfashion/mmfashion/datasets/Consumer_to_shop.py

Line 100 in 1467baf

if 'a' == 1:

Suboptimal results when running landmark detection demo

Hi,

I am planning to apply parts of the Deep Fashion project to one of my own and had some questions regarding your demo code.

When running the landmark detection demo code, the program returns insatisfactory results even when testing it on provided demo images.

Attached is the result when we ran the code on 07_1_front.jpg.

Is this how the pretrained demo model supposed to operate? And is it normal that the model returns different landmark points every iteration?

Thanks!

An error of the test of LandmarkDetector

When I ran the
python tools/test_landmark_detector.py --config configs/landmark_detect/landmark_detect_vgg.py --checkpoint checkpoint/LandmarkDetect/vgg/global/latest.pth ,
it occured:

Traceback (most recent call last):
File "tools/test_landmark_detector.py", line 84, in
main()
File "tools/test_landmark_detector.py", line 80, in main
logger=logger)
File "/home/user03/.conda/envs/open-mmlab/lib/python3.7/site-packages/mmfashion/apis/test_landmark_detector.py", line 36, in test_landmark_detector
_non_dist_test(model, dataset, cfg, validate=validate)
File "/home/user03/.conda/envs/open-mmlab/lib/python3.7/site-packages/mmfashion/apis/test_landmark_detector.py", line 63, in _non_dist_test
pred_vis, pred_lm, vis, landmark)
File "/home/user03/.conda/envs/open-mmlab/lib/python3.7/site-packages/mmfashion/core/evaluation/landmark_detect_eval.py", line 95, in evaluate_landmark_detection
vis * pred_lm_np, vis * landmark_np)
File "/home/user03/.conda/envs/open-mmlab/lib/python3.7/site-packages/mmfashion/core/evaluation/landmark_detect_eval.py", line 32, in compute_distance
if self.demo:
AttributeError: 'LandmarkDetectorEvaluator' object has no attribute 'demo'

How can I solve this problem? Thank you!

Using custom dataset

Thanks for your work.
I just want to figure out may I use my custom dataset for this project? I just crawled some data in the web, and I want to try this module with my dataset.

Demo for landmark detection

Thanks for your great work!
Is there a simple way to get one single img's landmark

What is the expected output for the VGG(Attribute Prediction) model on the demo images

These are the results I'm getting for the three demo images
1.
print
lace
knit
sleeve
floral

lace
knit
print
sleeve
striped

print
lace
knit
sleeve
striped

Which seems wrong
what is the suggested fix?

This is the command I am running
!python demo/test_predictor.py --input demo/imgs/attr_pred_demo3.jpg --checkpoint checkpoints/latest.pth

latest.pth is the VGG backbone global pooling model
and I have used list_attr_clothing.txt file provided on the deepfashion site

Best configuration for Attribute Prediction

What is the best configuration (config file, checkpoint) to do attribute prediction on an image, according to your experiments?

Installation error

Used pycharm to install the package and below is the error:

File "C:\Users\moham\PycharmProjects\mfashion1\venv\lib\site-packages\setuptools-40.8.0-py3.8.egg\setuptools\sandbox.py", line 45, in execfile
File "C:\Users\moham\AppData\Local\Temp\easy_install-4irumxcr\torch-0.1.2.post2\setup.py", line 225, in
File "C:\Users\moham\PycharmProjects\mfashion1\venv\lib\site-packages\setuptools-40.8.0-py3.8.egg\setuptools_init.py", line 145, in setup
File "C:\Python38\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python38\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "C:\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\moham\PycharmProjects\mfashion1\venv\lib\site-packages\setuptools-40.8.0-py3.8.egg\setuptools\command\bdist_egg.py", line 172, in run
File "C:\Users\moham\PycharmProjects\mfashion1\venv\lib\site-packages\setuptools-40.8.0-py3.8.egg\setuptools\command\bdist_egg.py", line 158, in call_command
File "C:\Python38\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\moham\PycharmProjects\mfashion1\venv\lib\site-packages\setuptools-40.8.0-py3.8.egg\setuptools\command\install_lib.py", line 11, in run
File "C:\Python38\lib\distutils\command\install_lib.py", line 107, in build
self.run_command('build_ext')
File "C:\Python38\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\moham\AppData\Local\Temp\easy_install-4irumxcr\torch-0.1.2.post2\setup.py", line 77, in run
"""Write version.py.
ModuleNotFoundError: No module named 'tools.cwrap'

Does not support windows

somewhere in this package require torch.distributed:

mmfashion/mmfashion/datasets/loader/sampler.py

Line 7 in 7a5acde

from torch.utils.data.distributed import DistributedSampler as _DistributedSampler

which is not supported by windows version of pytorch

Unexpected key when load pretrained model, attr_prob seems random.

Thank you for your great work!

I followed demo/test_predictor.py to load the model using:

model = build_predictor(cfg.model)
checkpoint = load_checkpoint(model, </path/to/resnet50/pretrained/model/latest.pth>, map_location='cpu')

And got:

The model and loaded state dict do not match exactly

unexpected key in source state_dict: roi_pool.linear.0.weight, roi_pool.linear.0.bias, concat.fc_fusion.weight, concat.fc_fusion.bias

The prediction on demo/attr_pred_demo1.jpg seems random since all probs are 0.4-0.6:

attr_prob = model(img_tensor, attr=None, landmark=None, return_loss=False)

attr_prob

tensor([[0.5356, 0.5116, 0.5132, 0.4532, 0.5143, 0.4566, 0.4571, 0.4589, 0.4532,
         0.5084, 0.4872, 0.4767, 0.4837, 0.5268, 0.3955, 0.4436, 0.5487, 0.4131,
         0.3939, 0.4990, 0.5329, 0.5224, 0.4197, 0.4822, 0.4122, 0.4306, 0.5518,
         0.4582, 0.5110, 0.4674, 0.4495, 0.4863, 0.4816, 0.5129, 0.5425, ...,
         0.4632]], device='cuda:0', grad_fn=<SigmoidBackward>)

The cfg.model is the default one:

cfg.model                                                                                                                                                  
Out[42]: 
{'type': 'GlobalPredictor',
 'backbone': {'type': 'ResNet'},
 'global_pool': {'type': 'GlobalPooling',
  'inplanes': (7, 7),
  'pool_plane': (2, 2),
  'inter_channels': [2048, 4096],
  'outchannels': 4096},
 'attr_predictor': {'type': 'AttrPredictor',
  'inchannels': 4096,
  'outchannels': 1000},
 'loss_attr': {'type': 'BCEWithLogitsLoss',
  'ratio': 1,
  'weight': None,
  'size_average': None,
  'reduce': None,
  'reduction': 'mean'},
 'pretrained': 'checkpoint/resnet50.pth'}

Any suggestion? What is the expected output for demo/attr_pred_demo1.jpg?

Thank you.

checkpoint/vgg16.pth

hi ，how can i get the init model checkpoint/vgg16.pth ???

How do you deal with consumer-to-shop clothing retrieval dataset?

Thank you for your excellent work.But I want to ask how do you split your dataset in consumer-to-shop retrieval experiment.(According to your paper deepfashionv1)In you note,you say that the gallery set here are all the shop images in "val + test" set.I don't know where is the query set when eval.Can you tell me some details on how to standard split consumer-to-shop dataset as in-shop dataset train\query\gallery. Thank you!

can't download models zoo from google drive

I can't download models zoo files from google drive because I'm in chinese mainland.
Would you give a resouce link of Baidu Cloud(pan.baidu.com) or 360 Cloud?
Thank you!

can't import init_weights_from

line 10 in tools/train_retriever.py :
...
from mmfashion.utils import init_weights_from
...

the import statement throw an exception since there is no init_weights_from under mmfashion.utils.

Retrievimg top-k images for consumer-to-shop task?

Thank you for the awesome dataset and code.

I have a question regarding one of your benchmarks, consumer-to-shop image retrieval.
I may be wrong, but in your paper, it seems like the FashionNet does not output the result for consumer-to-shop image retrieval.
Which similarity measure did you use to retrieve top-k consumer-to-shop images ?

Thanks a lot.

Content of "list_attr_cloth.txt"

What should the "list_attr_cloth.txt" file contain when performing the "test_predictor.py" on an image?

In order to test an image, I simply created an empty txt file, but it throws me a key error and I suspect the reason is that it expects something inside the txt.

Thank you.

loading pretrained model

Thank you for these awesome resources!
I downloaded the 'ResNet-50 | Global Pooling' .pth file from your model zoo to my_path.
I assume I could load the model like this:
model = ???
model.load_state_dict(torch.load(my_path, map_location='cpu')

However I do not know how to initialize the ResNet model structure (see the "???").
I failed with using:
import torchvision.models as models
model = models.resnet50(pretrained=False)

Could you specify how to load the model? That would help me a lot.

Quick start for making inference (e.g., attribute prediction, landmark detection) on an image?

Try to input an image and make inference (attributes/landmark) on an image using the pre-tained model.

Downloaded and installed the package

git clone https://github.com/open-mmlab/mmfashion.git
cd mmfashion
python setup.py install

Downloaded the pre-trained model checkpoint for attribute prediction, try to run some inference.

Q1: have to create a set of empty file data/Attr_Predict/Anno/test*.txt to run.

We should not be supposed to know the attribute when makes prediction on an unseen image, right? I guess the test here is testing, instead of inferencing?

mmfashion$ tree data
data
├── Attr_Predict
│   ├── Anno
│   │   ├── test_attr.txt
│   │   ├── test_bbox.txt
│   │   ├── test_cate.txt
│   │   ├── test_landmarks.txt
│   │   └── test.txt
│   └── Img
│       └── 11006215.jpg
├── prepare_attr_pred.py
├── prepare_in_shop.py
└── prepare_landmark_detect.py

Then, test_predictor.py gives error:

$ python tools/test_predictor.py --config configs/attribute_predict/roi_predictor_resnet_attr.py --checkpoint checkpoint/Predict/resnet/roi/latest.pth

2019-11-13 21:17:13,232 - INFO - Distributed test: False
/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/datasets/Attr_Pred.py:64: UserWarning: loadtxt: Empty input file: "data/Attr_Predict/Anno/test_attr.txt"
  self.labels = np.loadtxt(label_file, dtype=np.float32)
/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/datasets/Attr_Pred.py:77: UserWarning: loadtxt: Empty input file: "data/Attr_Predict/Anno/test_bbox.txt"
  self.bboxes = np.loadtxt(bbox_file, usecols=(0, 1, 2, 3))
/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/datasets/Attr_Pred.py:84: UserWarning: loadtxt: Empty input file: "data/Attr_Predict/Anno/test_landmarks.txt"
  self.landmarks = np.loadtxt(landmark_file)
dataset loaded
model built
load checkpoint from: checkpoint/Predict/resnet/roi/latest.pth
dataloader built
Total
/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/core/evaluation/attr_predict_eval.py:112: RuntimeWarning: invalid value encountered in double_scalars
  len(sorted_recall) - empty)
[Recall] top3 = nan, top5 = nan, top10 = nan
Traceback (most recent call last):
  File "tools/test_predictor.py", line 81, in <module>
    main()
  File "tools/test_predictor.py", line 77, in main
    logger=logger)
  File "/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/apis/test_predictor.py", line 36, in test_predictor
    _non_dist_test(model, dataset, cfg, validate=validate)
  File "/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/apis/test_predictor.py", line 67, in _non_dist_test
    attr_calculator.show_result()
  File "/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/core/evaluation/attr_predict_eval.py", line 160, in show_result
    self.compute_accuracy()
  File "/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/core/evaluation/attr_predict_eval.py", line 147, in compute_accuracy
    top['tp'], top['tn'])
  File "/home/bin/anaconda3/envs/mm/lib/python3.6/site-packages/mmfashion-0.1.0+ad2c911-py3.6.egg/mmfashion/core/evaluation/attr_predict_eval.py", line 141, in compute_one_accuracy
    accuracy[i] = float(tp[i] + tn[i]) / float(self.total)
ZeroDivisionError: float division by zero

No module named 'mmfashion'

After install the package successfully.

I run the test command
python tools/test_predictor.py
--config configs/attribute_predict/roi_predictor_vgg_attr.py
--checkpoint checkpoint/Predict/vgg/roi/latest.pth

I get the error: ModuleNotFoundError: No module named 'mmfashion'

But I check the pip list and find it contains the mmfashion:
mmfashion 0.1.0+27f4f53

All predictions are similar regardless images

I run demo/test_predictor.py with different my own images, including dress, shirt, pants. But no matter what image is tested, the predicted vector (attr_prob in the demo) is always similar. I calculated cosine similarity and it is always above 0.9 even comparing dress with pants.

config: configs/attribute_predict/global_predictor_vgg_attr.py
checkpoint: checkpoint/Predict/vgg16/global/latest.pth
pretrained: checkpoint/vgg16.pth OR the checkpoint above OR None

Torch version: 1.4.0+cu100

[Feature Request] Return the id of topK matches of a given query

Hi @veralauee, I'm thinking about checking the best matches suggested by the provided retriever, is there a way for me to access the retrieval results of a given query instead of just getting the topK recall? I think such a feature would be helpful for me to analysis the visual and semantic consistency of the query and topK suggestions, and provide useful guidance to the design of embedding extractor and similarity metric.
Much appreciated!

Python syntax error in configs/attribute_predict/roi_predictor_vgg_inshop.py

TokenError: EOF in multi-line statement can be found by running flake8 on the codebase.

Question about the top-k retrieval accuarcy

Hello, I have a question about the top-k retrieval accuarcy. In the paper, it says 'a successful retrieval is counted if the exact fashion item has been found in the top-k retrieved results.' and in the paper 'Where to buy it?' , top-k accuracy is defined as "the percentage of queries with at least one matching item retrieved within the first k results." Are they the same？
And in the released code, the topk retrieval accuarcy is calculated by the (tp) / (relevant_num) where tp means the num of exact fashion item has been found in the top-k retrieved results, the relevant_num means the num of cloth in gallery_set which has the same item_id as the query.
So I am confused,Can anyone give me some advice?

can't install "mmcv"

Doubt about Training attribute prediction

Can we train an attribute prediction model without the landmarks file?

This doubt also arises cause of the fact that here self.landmarks = None but here, in the get_basic_item function, origin_landmark = self.landmarks[idx] is used.

If at all we do really need the landmarks for training how do we save the predictions from the landmark detector into the format required for training attribute prediction model?

how to inference one image? just for clothes Detection

Error using resnet model TypeError: __init__() got an unexpected keyword argument 'layer_setting'

I got this error from below command. Similar command using VGG model works well.
Anyone could help? Thanks!!

python demo\test_predictor.py --input "demo\test_imgs\aaa.png" --config "configs/attribute_predict/global_predictor_resnet_attr.py" --checkpoint "checkpoint/Predict/resnet/global/model_best.pth"

Can't find: 'data/Attr_Predict/Anno/list_attr_cloth.txt'

Thanks for your awesome work!
Where can I find 'data/Attr_Predict/Anno/list_attr_cloth.txt'?
Thanks for your help!

Visualizing the generated landmarks

Hi,
I am able to predict the landmarks using your model.

Could I know how to go about visualizing the landmarks on the images
If the feature is not there how would you recommend I add it? ( Basic Open CV?)
Thanks

ImportError: cannot import name 'get_world_size' from 'torch.distributed'

env:
win10 cuda10.1

absl-py 0.8.0 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
absl-py 0.8.1
addict 2.2.1
apex 0.1
astor 0.8.0
attrs 19.3.0
backcall 0.1.0 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
bcrypt 3.1.7
beautifulsoup4 4.8.1
blas 1.0 mkl anaconda
bs4 0.0.1
bzip2 1.0.6 vc14_3 [vc14] anaconda
ca-certificates 2019.8.28 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
certifi 2019.9.11 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cffi 1.12.3 py37h7a1dbc1_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
chardet 3.0.4
Click 7.0
cloudpickle 1.2.2
colorama 0.4.1 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cryptography 2.8
cudatoolkit 10.1.243 h74a9793_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cycler 0.10.0 py37_0 anaconda
cython 0.29.13 py37ha925a31_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
decorator 4.4.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
detectron2 0.1
easydict 1.9
Flask 1.1.1
freetype 2.9.1 ha9979f8_1 anaconda
future 0.17.1 py37_0 anaconda
fvcore 0.1
gast 0.3.2
google-pasta 0.1.7
grpcio 1.24.1
grpcio 1.16.1 py37h351948d_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
h5py 2.10.0
hdf5 1.8.18 vc14h7a021fe_0 [] https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
html5lib 1.0.1
icc_rt 2019.0.0 h0cc432a_1 anaconda
icu 58.2 vc14hc45fdbb_0 [vc14] anaconda
idna 2.8
imagecorruptions 1.1.0
imageio 2.6.1
imgaug 0.2.6
intel-openmp 2019.5 281 anaconda
ipython 7.9.0 py37h39e3cac_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ipython_genutils 0.2.0 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
itsdangerous 1.1.0
jedi 0.15.1 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
Jinja2 2.10.3
joblib 0.14.0
jpeg 9b vc14h4d7706e_1 [vc14] anaconda
jsonpatch 1.24
jsonpointer 2.0
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.0
kiwisolver 1.1.0 py37ha925a31_0 anaconda
labelme 3.16.7
libopencv 3.4.1 hf0769c1_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libpng 1.6.37 h2a8f88b_0 anaconda
libprotobuf 3.9.2 h7bd577a_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libtiff 4.0.10 hb898794_2 anaconda
markdown 3.1.1 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
MarkupSafe 1.1.1
maskrcnn-benchmark 0.1
matplotlib 3.1.1 py37hc8f65d3_0 anaconda
mkl 2019.5 281 anaconda
mkl-service 2.3.0 py37hb782905_0 anaconda
mkl_fft 1.0.14 py37h14836fe_0 anaconda
mkl_random 1.1.0 py37h675688f_0 anaconda
mmcv 0.2.14
mmdet 1.0rc0+82c533b
mmfashion 0.1.0+0288d70
mmfashion 0.1.0+ad2c911
msvc_runtime 1.0.1 vc14_0 [vc14] anaconda
networkx 2.4
ninja 1.9.0.post1
ninja 1.9.0 py37h74a9793_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy 1.16.5 py37h19fb1c0_0 anaconda
numpy-base 1.16.5 py37hc3f5095_0 anaconda
olefile 0.46 py37_0 anaconda
opencv 3.4.1 py37h40b0b35_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
opencv-python 4.1.1.26
openssl 1.1.1d he774522_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandas 0.25.3
paramiko 2.6.0
parso 0.5.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pickleshare 0.7.5 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pillow 5.2.0 py37h08bbbbd_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
Pillow 6.2.1
pip 19.3.1
pip 19.2.3 py37_0 anaconda
portalocker 1.5.1
prompt_toolkit 2.0.10 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
protobuf 3.10.0
protobuf 3.9.2 py37h33f27b4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
py-opencv 3.4.1 py37hc319ecb_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pycocotools 2.0
pycparser 2.19 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pydensecrf 1.0rc2
pygments 2.4.2 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyhocon 0.3.51
PyNaCl 1.3.0
pyparsing 2.4.2 py_0 anaconda
pypinyin 0.35.4
pypiwin32 223
pyqt 5.9.2 py37ha878b3d_0 anaconda
python 3.7.1 h33f27b4_4 anaconda
python-dateutil 2.8.0 py37_0 anaconda
pytorch 1.3.0 py3.7_cuda101_cudnn7_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
pytz 2019.3 py_0 anaconda
PyWavelets 1.1.1
pywin32 225
PyYAML 5.1.2
pyzmq 18.1.0
qt 5.9.7 vc14h73c81de_0 [vc14] anaconda
QtPy 1.9.0
requests 2.22.0
scikit-image 0.16.2
scikit-learn 0.21.3
scipy 1.3.1
setuptools 41.4.0 py37_0 anaconda
sip 4.19.13 py37ha925a31_0 anaconda
six 1.12.0 py37_0 anaconda
sklearn 0.0
soupsieve 1.9.4
sqlite 3.30.0 he774522_0 anaconda
tb-nightly 2.1.0a20191016
tensorboard 2.0.0
tensorflow-gpu 2.0.0a0
termcolor 1.1.0
terminaltables 3.1.0
tf-estimator-nightly 1.14.0.dev2019030115
tk 8.6.7 vc14hb68737d_1 [] https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
torchfile 0.1.0
torchsummary 1.5.1
torchvision 0.4.1 py37_cu101 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
tornado 6.0.3 py37he774522_0 anaconda
tqdm 4.36.1 py_0 anaconda
traitlets 4.3.3 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
urllib3 1.25.6
vc 14.1 h0510ff6_4 anaconda
visdom 0.1.8.9
vs2015_runtime 14.16.27012 hf0eaf9b_0 anaconda
wcwidth 0.1.7 py37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
webencodings 0.5.1
websocket-client 0.56.0
werkzeug 0.16.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wheel 0.33.6 py37_0 anaconda
wincertstore 0.2 py37_0 anaconda
xz 5.2.4 h2fa13f4_4 anaconda
yacs 0.1.6
zlib 1.2.11 vc14h1cdd9ab_1 [vc14] anaconda
zstd 1.3.7 h508b16e_0 anaconda

cmd: python tools/test_landmark_detector.py --config configs/landmark_detect/landmark_detect_vgg.py --checkpoint Landmark_Detection_VGG-16.pth

When will you release demo for retriever_consumer_to_shop

Thank you for your contribution
I am looking forward to run a demo for retriever_consumer_to_shop, will you release that soon?

[Help wanted] Error when using my own custom split for in-shop clothes retrieval

@veralauee Hi, thanks for your effort for making things run smoothly. I have successfully evaluated the retriever on my machine with official query/gallery split. But when I evaluate the retriever on my own split I find the following error, of which I have got no clue. Can you help me with my issue or show me a workaround? Thanks in advance.

Problem description

(lingbo) lingbo.ylb@vg:~/projects/mmfashion$ sh retrieval.sh 
2019-12-19 10:42:25,501 - INFO - Distributed test: False
dataset loaded
2019-12-19 10:42:37,289 - INFO - load model from: /home/lingbo.ylb/.cache/torch/checkpoints/resnet50-19c8e357.pth
pretrained model /home/lingbo.ylb/.cache/torch/checkpoints/resnet50-19c8e357.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

model built
load checkpoint from: checkpoints/retrieval_clothes_resnet50/epoch_150.pth
Traceback (most recent call last):
  File "tools/test_retriever.py", line 88, in <module>
    main()
  File "tools/test_retriever.py", line 84, in main
    logger=logger)
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3.7.egg/mmfashion/apis/test_retriever.py", line 40, in test_retriever
    _non_dist_test(model, query_set, gallery_set, cfg, validate=validate)
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3.7.egg/mmfashion/apis/test_retriever.py", line 84, in _non_dist_test
    e.evaluate(query_embeds_np, gallery_embeds_np)
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3.7.egg/mmfashion/core/evaluation/retrieval_eval.py", line 85, in evaluate
    gallery_embeds, i)
  File "/home/lingbo.ylb/anaconda3/envs/lingbo/lib/python3.7/site-packages/mmfashion-0.1.0+7a5acde-py3.7.egg/mmfashion/core/evaluation/retrieval_eval.py", line 66, in single_query
    relevant_num = len(self.gallery_id2idx[query_id])
KeyError: 1801

Here is my retrieval script

python tools/test_retriever.py \
    --config configs/retriever/roi_retriever_resnet.py \
    --checkpoint checkpoints/retrieval_clothes_resnet50/epoch_150.pth

My guess is that there is certain rule for creating a valid query/gallery split, and I definitely violate some sort of rule here. Here I detail my preparation of custom split to help you replicate the error.

Preparation of my custom split

I am replicating an project on pose-guided human image generation, and somehow I wish to run a retrieval test using synthesized images to query from the real images. To this end, I simply set all the test images as "query" and all the train images as "gallery".

Here is my train/test split, and I think the partition is completely random, and by principle the ids in training split and testing split do not overlap, (but I haven't verified it yet). Could this be the reason why I cannot find a corresponding item for my query in the gallery?

test.txt
train.txt

After that I created an list_custom_patition.txt inside ./data/In-shop/Eval:
list_custom_partition.txt

Then I ran the prepare-in-shop.py script on the custom split file, and it all went well with no errors. Here are the generated annotations in a zip.
Anno.zip

How to replicate my error

I think by downloading the annotation files attached above and replace the custom file should do the trick. However, in case something went wrong with my preparation, here I attach my python script for generation list_custom_partition.txt from my train/test files.

################
# Step #1: Use PATN test as query and train as gallaery, replace MMFashion's default split.
# Potential bug:
#	For each test image there could be multiple synthesized results as query (using different source images)
#	In this case, I just copy the testing line twice/thrice and evaluate the topK-recall multiple times.
################

import os
import pandas

# you may want to change the prefix.
PREFIX = '/home/lingbo.ylb/projects/Pose-Transfer-mod/fashion_data'

# We don't need training lines.

# convert compact string to directory
# Input be like: fashionWOMENSkirtsid0000062904_3back.jpg
# Output be like: img/WOMEN/Skirts/id_00000629/04_3_back.jpg
def convert(dirty_str):
	s = dirty_str.strip('\n')[7:] # remove fashion
	if s.startswith('MEN'):
		gender='MEN'
		s = s[3:]
	else:
		gender='WOMEN'
		s = s[5:]

	t = s.find('id000')
	clothes_type = s[:t]
	s = s[t+2:]  # skip 'id'
	id_num = s[:8]  # id is eight digits
	s = s[8:]   # 04_3back.jpg
	delim = s[:4]   # 04_3
	ext = s[4:]

	return id_num, 'img/{}/{}/id_{}/{}_{}'.format(gender, clothes_type, id_num, delim, ext)

gallery_list = os.path.join(PREFIX, 'train.lst')
query_list = os.path.join(PREFIX, 'test.lst')

with open(gallery_list, 'r') as f:
	gallery_items = [convert(s) for s in f]

with open(query_list, 'r') as f:
	query_items = [convert(s) for s in f]

#print(query_items[0:10])
# print(query_items)
n = len(gallery_items) + len(query_items)

output = '/home/lingbo.ylb/projects/mmfashion/data/In-shop/Eval/list_custom_partition.txt'
with open(output, 'w') as f:
	f.write('%d\nimage_name item_id evaluation_status\n' % n)
	for g in gallery_items:
		f.write('{:<72s}{} gallery\n'.format(g[1],g[0]))
	for q in query_items:
		f.write('{:<72s}{} query\n'.format(q[1],q[0]))

Thanks again for your time!

Data Preparation Scrips Generate Wrong Landmarks Text

Hi,

When I used prepare_in_shop.py to split deepfashion dataset, I found that the generated landmarks.txt had wrong landmarks data.

1 1 117 067 144 065 073 219 183 210 0 0 0 0 0 0 
143 060 115 059 187 222 074 223 177 211 0 0 0 0 0 0 
155 090 086 245 0 0 0 0 0 0 0 0 0 0 0 0 
1 3 097 056 086 058 153 205 0 0 0 0 0 0 0 0 
3 3 160 063 105 060 168 112 092 104 157 148 104 145 0 0
...

The first two numbers of the 1,4,5 lines are 'clothes type' and 'variation type' but not landmarks.

Top-5 Recall vs Top-5 Acc.

@veralauee

Hello, can you kindly explain the difference between Top-5 Recall and Top-5 Acc. for attributes prediction?