Initialize a conda environment named texfit by running:
conda env create -f environment.yaml
conda activate texfit
# install mmcv and mmsegmentation
pip install -U openmim
mim install mmcv==1.2.1
mim install mmsegmentation==0.9.0
And then initialize an π€Accelerate environment with:
accelerate config
You need to download DFMM-Spotlight dataset from Google Drive and unzip to your own path /path/to/DFMM-Spotlight
. The dataset folder structure should be as follows:
DFMM-Spotlight
βββ train_images
βΒ Β βββ MEN-Denim-id_00000080-01_7_additional.png
βΒ Β βββ .......
βΒ Β βββ WOMEN-Tees_Tanks-id_00007979-04_4_full.png
βββ test_images
βΒ Β βββ MEN-Denim-id_00000089-03_7_additional.png
βΒ Β βββ .......
βΒ Β βββ WOMEN-Tees_Tanks-id_00007970-01_7_additional.png
βββ mask
βΒ Β βββ MEN-Denim-id_00000080-01_7_additional_mask_0.png
βΒ Β βββ .......
βΒ Β βββ WOMEN-Tees_Tanks-id_00007979-04_4_full_mask_0.png
βββ mask_ann
Β Β βββ train_ann_file.jsonl
Β Β βββ test_ann_file.jsonl
Important note: Replace all the /path/to
paths in the code and configuration files with real paths.
/path/to
paths exist in all the configuration files under the folder configs
and dataset/dfmm_spotlight_hf/dfmm_spotlight_hf.py
.
Train the editing region location module ERLM with the following command:
CUDA_VISIBLE_DEVICES=0 python train_erlm.py --opt ./configs/region_gen.yml
Train the local fashion image editing model TexFit with the following command:
bash train_texfit.sh
Once the ERLM and Texfit are trained, you can edit a fashion image locally by running the following command:
CUDA_VISIBLE_DEVICES=0 python pipeline.py \
--opt ./configs/region_gen.yml \
--img_path /path/to/your_fashion_image_path \
--output_path /path/to/edited_image_saving_path \
--text_prompt the_editing_text_prompt \
--erlm_model_path /path/to/trained_erlm_model_path \
--texfit_model_path /path/to/trained_texfit_model_path
For example:
CUDA_VISIBLE_DEVICES=0 python pipeline.py \
--opt ./configs/region_gen.yml \
--img_path examples/MEN-Denim-id_00000089-03_7_additional.png \
--output_path ./example_output.png \
--text_prompt 'denim blue lower clothing' \
--erlm_model_path experiments/region_gen/models/region_generation_epoch55.pth \
--texfit_model_path sd-model-finetuned/texfit-model
If you find this paper or the code useful for your research, please consider citing:
@inproceedings{wang2024texfit,
title={TexFit: Text-Driven Fashion Image Editing with Diffusion Models},
author={Wang, Tongxin and Ye, Mang},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={38},
number={9},
pages={10198--10206},
year={2024}
}
Our code is developed based on π€Diffusers and Text2Human. Thanks for their open source contributions.