Could you please give me an example of arranging ILSVRC2014 dataset? about openlongtailrecognition-oltr

[czjghost]

Hello, I am coming, about the dataset, first, you should download the original ILSVRC2012, specially, only downloading Training images (Task 1 & 2) and Validation images (all tasks) are enough:

In this project, you need to download the corresponding ImageNet-LT and Places-LT from here. Hence, the current structure of "./data" is:

|--data
    |--ImageNet_LT_open
        |--ILSVRC2010_val_00020390.JPEG
        ..
    |--ImageNet_LT_open.txt
    |--ImageNet_LT_test.txt
    |--ImageNet_LT_train.txt
    |--ImageNet_LT_val.txt
        ..

Then, we decompress files “ILSVRC2012_img_train.tar” and “ILSVRC2012_img_val.tar”. Note that, "ILSVRC2012_img_test.tar" is not used in this project. Now, we finish decompression (ILSVRC2012_img_train.tar -> train, ILSVRC2012_img_val.tar -> val_tar):

|--large_scale_dataset
    |--ImageNet
        |--train
        |--val_tar

Then, run the following code to generate test dataset (i.e., val):

import os
from shutil import copy

root = "./oltr/data/ImageNet_LT/ImageNet_LT_test.txt"
source_root = "./large_scale_dataset/ImageNet/val_tar"
target_root = "./large_scale_dataset/ImageNet/val"

with open(root, 'r') as f:
    while True:
        line = f.readline()
        if len(line) <= 1:
            break
        line = line.strip('\n')
        line = line.split(' ')[0]
        file_seq = line.split('/')
        pre_holder = file_seq[1]
        basename = file_seq[-1]
        target = target_root + "/" + pre_holder
        os.makedirs(target, exist_ok=True)
        source = source_root + "/" + basename
        os.makedirs(target, exist_ok=True)
        copy(source, target + "/" + basename)

Finally, you can see the structure in ""./large_scale_dataset/ImageNet/":

|--large_scale_dataset
    |--ImageNet
        |--train
        |--val_tar
        |--val

And the "val" holder( see the following image) is the corresponding test dataset, the "train" holder contains both the training and validation datasets.

I try to look back upon full process of generating those holders, if it lacks something, please tell me.

[Saba-Aly]

Hello did you get an idea about the data arrangements? As I am having problem in validation dataset.

[Xglbrilliant]

Hello, can you provide the link to download the imagenet2014 dataset?

[czjghost]

Hello, can you provide the link to download the imagenet2014 dataset?

https://image-net.org/challenges/LSVRC/2014/2014-downloads.php

[czjghost]

I also have problem about the arrangement of ILSVRC2014 in this project, it seems that author maybe use ILSVRC2012 rather than ILSVRC2014 ?

update 2023.4.20:
By downloading the ILSVRC2012 and splitting test dataset into the format in "ImageNet_LT_test.txt", the code runs successfully.

dataset holder

|--ImageNet
    |--train
        |--n01440764
        ..
    |--val
        |--n01440764
        ..

Note that, places365 keeps the original file structure, and renaming the holder is enough.

[butcher1226]

I also have problem about the arrangement of ILSVRC2014 in this project, it seems that author maybe use ILSVRC2012 rather than ILSVRC2014 ?

update 2023.4.20: By downloading the ILSVRC2012 and splitting test dataset into the format in "ImageNet_LT_test.txt", the code runs successfully.

dataset holder

|--ImageNet
    |--train
        |--n01440764
        ..
    |--val
        |--n01440764
        ..

Note that, places365 keeps the original file structure, and renaming the holder is enough.

Sorry to bother, could you tell how to split test dataset into the format in "ImageNet_LT_test.txt", since ILSVRC2012_img_test_v10102019.tar doesn't have the arctecture like test/n01440764

[W-monster]

我对这个项目中 ILSVRC2014 的安排也有疑问，似乎作者可能使用 ILSVRC2012 而不是 ILSVRC2014 ？
2023.4.20更新：通过下载ILSVRC2012并将测试数据集拆分为“ImageNet_LT_test.txt”中的格式，代码运行成功。
数据集持有者

|--ImageNet
    |--train
        |--n01440764
        ..
    |--val
        |--n01440764
        ..

我对这个项目中 ILSVRC2014 的安排也有疑问，似乎作者可能使用 ILSVRC2012 而不是 ILSVRC2014 ？

2023.4.20更新： 通过下载ILSVRC2012并将测试数据集拆分为“ImageNet_LT_test.txt”中的格式，代码运行成功。

数据集持有者

|--ImageNet
    |--train
        |--n01440764
        ..
    |--val
        |--n01440764
        ..

请注意，places365 保留了原始文件结构，重命名 holder
你好，打扰下，能分享下你是如何将数据集拆分成如下结构的吗
|--ImageNet
|--train
|--n01440764
..
|--val
|--n01440764
..