tekeburak / trendyol-cv-case

Trendyol Image Similarity

Trendyol Image Similarity

Project Description:

This project focuses on measuring the similarity between two products.

The provided JSON file includes 20,000 fashion product images along with their bounding boxes. The goal is to identify the most similar fashion products within this dataset for a given input image.

The task involves randomly selecting 10 cropped images from the JSON file and identifying the 12 images that are most similar to each of them.

Installation Instructions:

To install the required dependencies, run the following command:

pip install -r requirements.txt

Usage Instructions:

1. Specify the necessary paths in main.py:

   • images_folder: Path to the folder where images will be saved.
   • export_json_path: Path to the JSON file containing image and bounding box information.
   • crop_out_folder: Path to the folder where cropped images will be saved.
   • dataset_csv_out: Path to the CSV file for extracting dataset information.

2. Run the main.py script:

python main.py

Features and Functionalities:

This project utilizes the following models for feature extraction:

• ResNet50 with ImageNet pretrained weights.
• ViT with DeepFashion In-shop pretrained weights using Open Metric Learning (OML) library.
• ViT with Stanford Online Products pretrained weights using OML.
• For efficient similarity search, Faiss library is used on ResNet feature vectors, saving feature vectors as indexes and allowing fast search query vectors.

Based on the results, ViT with DeepFashion In-shop pretrained weights outperforms other models.

Example Code Snippets:

from utils.image_cropper import ImageCropper
from utils.image_downloader import ImageDownloader
from utils.resnet_inference import ImageSimilarityResNet
from utils.vit_inference import ImageSimilarityViT


def main():
    
    images_folder = "data/images"
    export_json_path = "data/export.json"
    crop_out_folder = "data/cropped_images"
    dataset_csv_out = "data/dataset_info.csv"
    
    downloader = ImageDownloader(images_folder,
                                 export_json_path)
    downloader.download_images()

    cropper = ImageCropper(images_folder,
                           export_json_path,
                           crop_out_folder,
                           dataset_csv_out)
    cropper.crop_images()
    
    img_sim_resnet = ImageSimilarityResNet(crop_out_folder,
                                           dataset_csv_out,
                                           "faiss")
    img_sim_resnet.analyze_similarity()
    
    img_sim_vit = ImageSimilarityViT(crop_out_folder,
                                     dataset_csv_out,
                                     pretrained_dataset="StanfordOnline")
    img_sim_vit.analyze_similarity()

if __name__ == "__main__":
    main()

Results:

The outcomes of all models have been documented in PDF files, accessible in the results folder of this repository. Each PDF contains the query image and its top 12 similar images, providing a visual representation of the image retrieval process. You can click on the links below to view the PDF results:

• Result for query 1 image using ResNet(ImageNet) | Result for query 2 image using ResNet(ImageNet)

• Result for query 1 image using ResNet(ImageNet) along with Faiss | Result for query 2 image using ResNet(ImageNet) along with Faiss

• Result for query 1 image using ViT(DeepFashion In-shop) | Result for query 2 image using ViT(DeepFashion In-shop)

• Result for query 1 image using ViT(Stanford Online Products) | Result for query 2 image using ViT(Stanford Online Products)

Feel free to explore the similarity assessments achieved by each model.

License Information:

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License.

For more details, please refer to the LICENSE file.