janghyuncho / picie Goto Github PK

Hi @janghyuncho . Thanks for the great research.
For only stuff or only thing , I wanna know the dataset setting which in your comparison of the performance in Table 2 or Table 4 in your paper.
Is your setting the same with IIC?, like if want to train for only stuff , and the number of stuff catergory is 15,
so set the K is 15, and mask the those objects belong to thing in all the images in datasets?
Thanks.

Hi @janghyuncho . Thanks for the great research.

When I want to try the visualize.ipynb. I got this error about fpn.

Can you fix it?
Thanks.

I don't understand if it's a unsupervised learning，why I need annotations. When I delete the annotations，the code will be wrong. I think if it's a unsupervised learning, does the dataset just need visual image?

Dear @janghyuncho ,

thank you very much for your work. Would it please be possible to also shared the other baselines (IIC, modified DC) trained on Cityscapes? It would be of a great help.

Thank you very much in advance.

The link to the cocostuff annotation png files (http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/stuffthingmaps_trainval2017.zip) seems dead. Do you have any knowledge of a mirror where these could be downloaded ?
I tried merging COCO stuff and things annotations myself, but my reproductions of PiCIE results are still off by a few points, therefore I suspect the annotations must not be perfectly right yet.

Thanks for sharing such great work.
I was gonna try to run visualize.ipynb. To do that, I run

python prepare_histogram.py --data_root datasets --save_root results --stuff --thing

But I faced this error

Could you help me?

And if I want to get labels(masks) for unlabeled images, what should I run?

Thanks in advance.

Hello， I want to construct the coco dataset as yours, but there are 6 json files in annotations/, which two is needed? Please tell me, thinks!

Hi,

Thank you for your inspiring work. I am also working on self-supervised approaches for semantic segmentation. I hope to have a fair comparison with your work on Cityscapes. However, I could not find enough information to reproduce the best results in the paper and this repo. Could you provide your models trained on Cityscapes? I'd highly appreciate some help.

Hello @janghyuncho !

Your research is amazing, great job :)

I have been working with it, but when I run:

python prepare_histogram.py --data_root datasets --save_root results --stuff --thing

I get the following error:

C: ...\PiCIE\prepare_histogram.py", line 51, in
args = get_args()
NameError: name 'get_args' is not defined

Am I missing something or should I change something in the prepare_hsitogram.py?
I am guessing that function is not called, but I can't figure out where is located, so I can add it, any ideas?

Thanks for your time!

There may be a type in line 126 in transform_inv function. The entry in inv_list should be "grey" not "gray".

When I train the model, the performance is lower than the paper.
I think you also experimented with many models to get the best model.
I'm doing additional training now, and it would be very helpful if you could provide the results for previously trained models.

Can you provide the mean and standard deviation of the performance of the previously run models?
Can you provide performance for stuff only and thing only for models that do not use auxiliary clustering (+H)?

Thanks.

This is my conda environment for RTX 3090 GPU that supports CUDA>=11.0.
python=3.7 -> 3.6
torch=1.2.0 -> 1.8.1
torchvision=0.4.0 -> 0.9.1
cudatoolkit=10.0 -> 11.0
faiss-gpu=1.6.3 -> 1.7.1

I trained 2 models using the train_picie.sh.

- stuff and things
All ACC : (1st) 39.3131 (± 16.0990) (2nd) 44.2331 (± 19.2532)
m IOU : (1st) 9.0956 (2nd) 10.9407

mean All ACC : 41.7731 ± 2.4600
best All ACC : 44.2331 <---> 48.09 (paper), 49.99 (paper, +H)
mean mIOU : 10.0182 ± 0.9226
best mIOU : 10.9407 <---> 13.84 (paper), 14.36 (paper, +H)

- stuff
All ACC : (1st) 42.6251 (± 22.3882) (2nd) 48.9712 (± 27.5140)
m IOU : (1st) 7.8613 (2nd) 9.9796

mean All ACC : 45.7982 ± 3.1731
best All ACC : 48.9712 <---> 74.56 (paper, +H)
mean mIOU : 8.9205 ± 1.0592
best mIOU : 9.9796 <---> 17.32 (paper, +H)

- thing
All ACC : (1st) 32.7959 (± 17.7133) (2nd) 47.6261 (± 26.1709)
m IOU : (1st) 4.7463 (2nd) 7.2779

mean All ACC : 40.2110 ± 7.4150
best All ACC : 47.6261 <---> 69.39 (paper, +H)
mean mIOU : 6.0121 ± 1.2658
best mIOU : 7.2779 <---> 23.83 (paper, +H)

Additionally, there are minor mistakes.

1. It is not working for evaluation of stuff only. (I modified it by using the thing only part.)
   error part : link

2. 'is_first' option is not pre-defined. (first_batch?)
   error part : link
   definition : link

3. test bash code also require 'save_root'.

I have a dataset which contains aboout 300 pictures, how can I use my one datasets to train this model?
And How can I load the pre-trained model?

Hi Jang,

Thank you for your great work.

I was running the code on my own unlabeled dataset.

Is there a way to obtain segmentation maps of my training dataset?

When I try to untar the archive using tar -xvf picie_coco.tar , I get this error:
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

I have also tried to extract with archive manager, without a terminal

Hi, thanks for your code.
I have an issue, in the training processing, we need to do the same transformation for the two input:
input1 = eqv_transform_if_needed(args, dataloader, indice, input1.cuda(non_blocking=True))
featmap1 = model(input1)

    input2 = input2.cuda(non_blocking=True)
    featmap2 = eqv_transform_if_needed(args, dataloader, indice, model(input2))

however, when the input2 image is passed from the FPN model, it generates different dimensional output compared to the input1, how to make sure that these two transformations have the same effects?

Hello, thanks for your great work. And now, I am trying to train the model in my device. Can you tell me how many GPUs you used to train the model?

Hello Janghyuncho,

Thank you for your work. I am wondering why is it necessary to compute an histogram_train and histogram_valid for visualization?

Hi:

Thanks for sharing this great work! Recently I try to adapt this work to the task of training PiCIE on MS-COCO with full class categories without merging them into superclasses (i.e. 12 things + 15 stuff), by doing the following modifications:

1. Re-arrage the class-ids of ground-truth label to keep the consecutiveness by mapping indices 0-90 into 0-79, and 91-181 into 80-170, instead of merging them into 0-26 using 'fine_to_coarse_dict.pickle'.
2. Set both K_train and K_test to 171 in train_picie.sh and keep the rest hyperparameters the same as ./sh_files/train_picie.sh.

here are my train logs:

2021-10-09 00:26:42,166:INFO: Namespace(K_stuff=91, K_test=171, K_things=80, K_train=171, X=80, arch='resnet18', augment=True, batch_size_cluster=256, batch_size_test=128, batch_size_train=128, blur=True, comment='', data_root='datasets/coco/', equiv=True, eval_only=False, eval_path=None, fullcoco=False, grey=True, h_flip=True, in_dim=128, jitter=True, kmeans_n_iter=20, lr=0.0001, metric_test='cosine', metric_train='cosine', min_scale=0.5, momentum=0.9, mse=False, no_balance=False, no_merge_coco=True, num_batches=1, num_epoch=10, num_init_batches=20, num_workers=4, optim_type='Adam', pretrain=True, random_crop=True, repeats=1, res=320, res1=320, res2=640, restart=False, restart_path=None, save_eval_path='results/picie/train/2/augmented/res1=320_res2=640/jitter=True_blur=True_grey=True/equiv/h_flip=True_v_flip=False_crop=True/min_scale\=0.5/K_train=171_cosine/K_test=171_cosine', save_model_path='results/picie/train/2/augmented/res1=320_res2=640/jitter=True_blur=True_grey=True/equiv/h_flip=True_v_flip=False_crop=True/min_scale\=0.5/K_train=171_cosine', save_root='results/picie/train/2/augmented/res1=320_res2=640/jitter=True_blur=True_grey=True/equiv/h_flip=True_v_flip=False_crop=True/min_scale\=0.5', seed=1, stuff=True, thing=True, v_flip=False, val_type='train', version=7, weight_decay=0.0005)
2021-10-09 00:26:47,642:INFO: ====== METRIC TEST : cosine ======

2021-10-09 00:26:53,735:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 00:26:53,736:INFO: Batch label size : [128, 320, 320]
2021-10-09 00:26:53,736:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 00:26:54,537:INFO: 0/17
2021-10-09 00:27:10,761:INFO: ACC - All: 9.8964
2021-10-09 00:27:10,762:INFO: mIOU - All: 1.5880
2021-10-09 00:27:10,762:INFO: ACC - Thing: 17.9023
2021-10-09 00:27:10,762:INFO: mIOU - Thing: 3.0928
2021-10-09 00:27:10,762:INFO: ACC - Stuff: 17.3441
2021-10-09 00:27:10,762:INFO: mIOU - Stuff: 3.0644
2021-10-09 00:27:11,293:INFO:
============================= [Epoch 0] =============================

2021-10-09 00:27:11,293:INFO: Start computing centroids.
2021-10-09 00:27:34,421:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 00:27:34,421:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 00:27:35,483:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 00:31:38,532:INFO: Initial k-means loss: 0.4483
2021-10-09 00:38:18,528:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.4512
2021-10-09 00:45:39,889:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 00:45:39,890:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 00:45:40,660:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 00:49:32,051:INFO: Initial k-means loss: 0.4343
2021-10-09 00:56:00,442:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.4384
2021-10-09 01:02:49,640:INFO: -Centroids ready. [Loss: 0.45128| 0.43836/ Time: 0:0:35:38]

2021-10-09 01:03:10,389:INFO: Centroid size : [171, 128]
2021-10-09 01:03:10,389:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 01:03:10,390:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 01:03:13,166:INFO: [Assigning labels] 0 / 194
2021-10-09 01:18:31,839:INFO: Centroid size : [171, 128]
2021-10-09 01:18:31,840:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 01:18:31,840:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 01:18:33,925:INFO: [Assigning labels] 0 / 194
2021-10-09 01:31:45,003:INFO: -Cluster labels ready. [0:0:28:56]

2021-10-09 01:31:45,007:INFO: Start training ...
2021-10-09 01:32:02,046:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 01:32:02,047:INFO: Batch label size : [128, 80, 80]
2021-10-09 01:32:02,047:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 01:32:02,929:INFO: 0 / 388
2021-10-09 01:45:10,229:INFO: 200 / 388
2021-10-09 01:57:20,011:INFO: ====== METRIC TEST : cosine ======

2021-10-09 01:57:25,053:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 01:57:25,053:INFO: Batch label size : [128, 320, 320]
2021-10-09 01:57:25,053:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 01:57:25,869:INFO: 0/17
2021-10-09 01:57:42,822:INFO: ACC - All: 24.2850
2021-10-09 01:57:42,823:INFO: mIOU - All: 7.1373
2021-10-09 01:57:42,823:INFO: ACC - Thing: 38.0029
2021-10-09 01:57:42,823:INFO: mIOU - Thing: 12.7254
2021-10-09 01:57:42,824:INFO: ACC - Stuff: 38.6157
2021-10-09 01:57:42,824:INFO: mIOU - Stuff: 13.0482
2021-10-09 01:57:42,832:INFO: ====== METRIC TEST : cosine ======

2021-10-09 01:57:47,818:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 01:57:47,819:INFO: Batch label size : [128, 320, 320]
2021-10-09 01:57:47,819:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 01:57:48,595:INFO: 0/17
2021-10-09 01:58:05,249:INFO: ACC - All: 27.1829
2021-10-09 01:58:05,250:INFO: mIOU - All: 7.6102
2021-10-09 01:58:05,250:INFO: ACC - Thing: 41.4974
2021-10-09 01:58:05,250:INFO: mIOU - Thing: 12.1366
2021-10-09 01:58:05,250:INFO: ACC - Stuff: 38.8955
2021-10-09 01:58:05,250:INFO: mIOU - Stuff: 14.1767
2021-10-09 01:58:05,259:INFO: ============== Epoch [0] ==============
2021-10-09 01:58:05,259:INFO: Time: [0:1:30:54]
2021-10-09 01:58:05,259:INFO: K-Means loss : 0.45128 | 0.43836
2021-10-09 01:58:05,259:INFO: Training Total Loss : 4.83569
2021-10-09 01:58:05,259:INFO: Training CE Loss (Total | Within | Across) : 4.83569 | 4.81816 | 4.85321
2021-10-09 01:58:05,259:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 01:58:05,259:INFO: [View 1] ACC: 24.2850 | mIoU: 7.1373
2021-10-09 01:58:05,259:INFO: [View 2] ACC: 27.1829 | mIoU: 7.6102
2021-10-09 01:58:05,259:INFO: ========================================

2021-10-09 01:58:06,719:INFO:
============================= [Epoch 1] =============================

2021-10-09 01:58:06,719:INFO: Start computing centroids.
2021-10-09 01:58:26,708:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 01:58:26,709:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 01:58:27,838:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 02:02:34,372:INFO: Initial k-means loss: 0.2232
2021-10-09 02:09:18,356:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.2253
2021-10-09 02:16:39,888:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 02:16:39,889:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 02:16:40,660:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 02:20:36,539:INFO: Initial k-means loss: 0.2123
2021-10-09 02:27:04,407:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.2143
2021-10-09 02:33:55,241:INFO: -Centroids ready. [Loss: 0.22525| 0.21440/ Time: 0:0:35:49]

2021-10-09 02:34:15,001:INFO: Centroid size : [171, 128]
2021-10-09 02:34:15,002:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 02:34:15,002:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 02:34:18,144:INFO: [Assigning labels] 0 / 194
2021-10-09 02:49:25,416:INFO: Centroid size : [171, 128]
2021-10-09 02:49:25,417:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 02:49:25,417:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 02:49:27,891:INFO: [Assigning labels] 0 / 194
2021-10-09 03:02:35,559:INFO: -Cluster labels ready. [0:0:28:40]

2021-10-09 03:02:35,563:INFO: Start training ...
2021-10-09 03:02:53,373:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 03:02:53,374:INFO: Batch label size : [128, 80, 80]
2021-10-09 03:02:53,374:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 03:02:54,179:INFO: 0 / 388
2021-10-09 03:15:55,349:INFO: 200 / 388
2021-10-09 03:27:54,270:INFO: ====== METRIC TEST : cosine ======

2021-10-09 03:27:59,453:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 03:27:59,453:INFO: Batch label size : [128, 320, 320]
2021-10-09 03:27:59,453:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 03:28:00,256:INFO: 0/17
2021-10-09 03:28:17,828:INFO: ACC - All: 22.7207
2021-10-09 03:28:17,829:INFO: mIOU - All: 7.3259
2021-10-09 03:28:17,829:INFO: ACC - Thing: 32.4342
2021-10-09 03:28:17,829:INFO: mIOU - Thing: 13.8477
2021-10-09 03:28:17,829:INFO: ACC - Stuff: 35.6936
2021-10-09 03:28:17,829:INFO: mIOU - Stuff: 11.8657
2021-10-09 03:28:17,840:INFO: ====== METRIC TEST : cosine ======

2021-10-09 03:28:22,962:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 03:28:22,962:INFO: Batch label size : [128, 320, 320]
2021-10-09 03:28:22,962:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 03:28:23,765:INFO: 0/17
2021-10-09 03:28:40,975:INFO: ACC - All: 23.1472
2021-10-09 03:28:40,976:INFO: mIOU - All: 7.2321
2021-10-09 03:28:40,977:INFO: ACC - Thing: 38.1891
2021-10-09 03:28:40,977:INFO: mIOU - Thing: 14.3081
2021-10-09 03:28:40,977:INFO: ACC - Stuff: 35.3766
2021-10-09 03:28:40,977:INFO: mIOU - Stuff: 10.9369
2021-10-09 03:28:40,987:INFO: ============== Epoch [1] ==============
2021-10-09 03:28:40,987:INFO: Time: [0:1:30:34]
2021-10-09 03:28:40,987:INFO: K-Means loss : 0.22525 | 0.21440
2021-10-09 03:28:40,987:INFO: Training Total Loss : 4.39373
2021-10-09 03:28:40,987:INFO: Training CE Loss (Total | Within | Across) : 4.39373 | 4.36556 | 4.42191
2021-10-09 03:28:40,987:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 03:28:40,987:INFO: [View 1] ACC: 22.7207 | mIoU: 7.3259
2021-10-09 03:28:40,987:INFO: [View 2] ACC: 23.1472 | mIoU: 7.2321
2021-10-09 03:28:40,987:INFO: ========================================

2021-10-09 03:28:42,444:INFO:
============================= [Epoch 2] =============================

2021-10-09 03:28:42,444:INFO: Start computing centroids.
2021-10-09 03:29:03,685:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 03:29:03,686:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 03:29:04,706:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 03:33:12,056:INFO: Initial k-means loss: 0.1106
2021-10-09 03:39:53,673:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.1097
2021-10-09 03:47:17,752:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 03:47:17,753:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 03:47:18,500:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 03:51:12,067:INFO: Initial k-means loss: 0.1047
2021-10-09 03:57:38,881:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.1043
2021-10-09 04:04:33,968:INFO: -Centroids ready. [Loss: 0.11023| 0.10459/ Time: 0:0:35:51]

2021-10-09 04:04:55,258:INFO: Centroid size : [171, 128]
2021-10-09 04:04:55,259:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 04:04:55,259:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 04:04:57,977:INFO: [Assigning labels] 0 / 194
2021-10-09 04:20:03,796:INFO: Centroid size : [171, 128]
2021-10-09 04:20:03,797:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 04:20:03,797:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 04:20:06,492:INFO: [Assigning labels] 0 / 194
2021-10-09 04:33:18,896:INFO: -Cluster labels ready. [0:0:28:45]

2021-10-09 04:33:18,900:INFO: Start training ...
2021-10-09 04:33:36,001:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 04:33:36,002:INFO: Batch label size : [128, 80, 80]
2021-10-09 04:33:36,002:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 04:33:36,807:INFO: 0 / 388
2021-10-09 04:46:30,782:INFO: 200 / 388
2021-10-09 04:58:34,859:INFO: ====== METRIC TEST : cosine ======

2021-10-09 04:58:40,046:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 04:58:40,046:INFO: Batch label size : [128, 320, 320]
2021-10-09 04:58:40,047:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 04:58:40,854:INFO: 0/17
2021-10-09 04:58:58,391:INFO: ACC - All: 24.5717
2021-10-09 04:58:58,392:INFO: mIOU - All: 6.6422
2021-10-09 04:58:58,393:INFO: ACC - Thing: 38.2366
2021-10-09 04:58:58,393:INFO: mIOU - Thing: 12.8897
2021-10-09 04:58:58,393:INFO: ACC - Stuff: 41.9975
2021-10-09 04:58:58,393:INFO: mIOU - Stuff: 11.9853
2021-10-09 04:58:58,402:INFO: ====== METRIC TEST : cosine ======

2021-10-09 04:59:03,479:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 04:59:03,479:INFO: Batch label size : [128, 320, 320]
2021-10-09 04:59:03,479:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 04:59:04,301:INFO: 0/17
2021-10-09 04:59:21,282:INFO: ACC - All: 25.1037
2021-10-09 04:59:21,282:INFO: mIOU - All: 7.0031
2021-10-09 04:59:21,283:INFO: ACC - Thing: 48.4587
2021-10-09 04:59:21,283:INFO: mIOU - Thing: 13.5206
2021-10-09 04:59:21,283:INFO: ACC - Stuff: 35.8929
2021-10-09 04:59:21,283:INFO: mIOU - Stuff: 11.7855
2021-10-09 04:59:21,292:INFO: ============== Epoch [2] ==============
2021-10-09 04:59:21,293:INFO: Time: [0:1:30:39]
2021-10-09 04:59:21,293:INFO: K-Means loss : 0.11023 | 0.10459
2021-10-09 04:59:21,293:INFO: Training Total Loss : 4.30574
2021-10-09 04:59:21,293:INFO: Training CE Loss (Total | Within | Across) : 4.30574 | 4.28338 | 4.32810
2021-10-09 04:59:21,293:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 04:59:21,293:INFO: [View 1] ACC: 24.5717 | mIoU: 6.6422
2021-10-09 04:59:21,293:INFO: [View 2] ACC: 25.1037 | mIoU: 7.0031
2021-10-09 04:59:21,293:INFO: ========================================

2021-10-09 04:59:22,772:INFO:
============================= [Epoch 3] =============================

2021-10-09 04:59:22,772:INFO: Start computing centroids.
2021-10-09 04:59:43,369:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 04:59:43,370:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 04:59:44,473:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 05:03:51,557:INFO: Initial k-means loss: 0.0755
2021-10-09 05:10:37,660:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0761
2021-10-09 05:18:03,777:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 05:18:03,778:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 05:18:04,573:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 05:22:02,173:INFO: Initial k-means loss: 0.0722
2021-10-09 05:28:35,818:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0727
2021-10-09 05:35:33,506:INFO: -Centroids ready. [Loss: 0.07620| 0.07283/ Time: 0:0:36:11]

2021-10-09 05:35:54,508:INFO: Centroid size : [171, 128]
2021-10-09 05:35:54,509:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 05:35:54,509:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 05:35:57,352:INFO: [Assigning labels] 0 / 194
2021-10-09 05:51:12,051:INFO: Centroid size : [171, 128]
2021-10-09 05:51:12,052:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 05:51:12,052:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 05:51:14,680:INFO: [Assigning labels] 0 / 194
2021-10-09 06:04:25,581:INFO: -Cluster labels ready. [0:0:28:52]

2021-10-09 06:04:25,585:INFO: Start training ...
2021-10-09 06:04:42,483:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 06:04:42,484:INFO: Batch label size : [128, 80, 80]
2021-10-09 06:04:42,484:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 06:04:43,391:INFO: 0 / 388
2021-10-09 06:17:43,126:INFO: 200 / 388
2021-10-09 06:29:43,005:INFO: ====== METRIC TEST : cosine ======

2021-10-09 06:29:48,256:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 06:29:48,256:INFO: Batch label size : [128, 320, 320]
2021-10-09 06:29:48,256:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 06:29:49,093:INFO: 0/17
2021-10-09 06:30:06,524:INFO: ACC - All: 24.1964
2021-10-09 06:30:06,525:INFO: mIOU - All: 6.4970
2021-10-09 06:30:06,525:INFO: ACC - Thing: 45.5011
2021-10-09 06:30:06,526:INFO: mIOU - Thing: 12.9584
2021-10-09 06:30:06,526:INFO: ACC - Stuff: 30.5258
2021-10-09 06:30:06,526:INFO: mIOU - Stuff: 9.8375
2021-10-09 06:30:06,534:INFO: ====== METRIC TEST : cosine ======

2021-10-09 06:30:11,667:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 06:30:11,667:INFO: Batch label size : [128, 320, 320]
2021-10-09 06:30:11,667:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 06:30:12,478:INFO: 0/17
2021-10-09 06:30:29,577:INFO: ACC - All: 24.7228
2021-10-09 06:30:29,578:INFO: mIOU - All: 6.6396
2021-10-09 06:30:29,578:INFO: ACC - Thing: 40.2358
2021-10-09 06:30:29,578:INFO: mIOU - Thing: 13.2917
2021-10-09 06:30:29,579:INFO: ACC - Stuff: 40.0240
2021-10-09 06:30:29,579:INFO: mIOU - Stuff: 10.6481
2021-10-09 06:30:29,589:INFO: ============== Epoch [3] ==============
2021-10-09 06:30:29,589:INFO: Time: [0:1:31:7]
2021-10-09 06:30:29,589:INFO: K-Means loss : 0.07620 | 0.07283
2021-10-09 06:30:29,589:INFO: Training Total Loss : 4.26950
2021-10-09 06:30:29,589:INFO: Training CE Loss (Total | Within | Across) : 4.26950 | 4.25079 | 4.28820
2021-10-09 06:30:29,589:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 06:30:29,589:INFO: [View 1] ACC: 24.1964 | mIoU: 6.4970
2021-10-09 06:30:29,589:INFO: [View 2] ACC: 24.7228 | mIoU: 6.6396
2021-10-09 06:30:29,589:INFO: ========================================

2021-10-09 06:30:31,229:INFO:
============================= [Epoch 4] =============================

2021-10-09 06:30:31,229:INFO: Start computing centroids.
2021-10-09 06:30:52,607:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 06:30:52,608:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 06:30:53,638:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 06:34:58,272:INFO: Initial k-means loss: 0.0618
2021-10-09 06:41:42,564:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0622
2021-10-09 06:49:07,210:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 06:49:07,211:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 06:49:07,956:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 06:53:02,606:INFO: Initial k-means loss: 0.0572
2021-10-09 06:59:36,551:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0579
2021-10-09 07:06:33,523:INFO: -Centroids ready. [Loss: 0.06225| 0.05798/ Time: 0:0:36:2]

2021-10-09 07:06:53,853:INFO: Centroid size : [171, 128]
2021-10-09 07:06:53,854:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 07:06:53,854:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 07:06:56,900:INFO: [Assigning labels] 0 / 194
2021-10-09 07:22:10,938:INFO: Centroid size : [171, 128]
2021-10-09 07:22:10,940:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 07:22:10,940:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 07:22:13,475:INFO: [Assigning labels] 0 / 194
2021-10-09 07:35:22,684:INFO: -Cluster labels ready. [0:0:28:49]

2021-10-09 07:35:22,688:INFO: Start training ...
2021-10-09 07:35:39,805:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 07:35:39,806:INFO: Batch label size : [128, 80, 80]
2021-10-09 07:35:39,806:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 07:35:40,611:INFO: 0 / 388
2021-10-09 07:48:35,877:INFO: 200 / 388
2021-10-09 08:00:38,165:INFO: ====== METRIC TEST : cosine ======

2021-10-09 08:00:43,357:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 08:00:43,358:INFO: Batch label size : [128, 320, 320]
2021-10-09 08:00:43,358:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 08:00:44,174:INFO: 0/17
2021-10-09 08:01:01,751:INFO: ACC - All: 24.4110
2021-10-09 08:01:01,752:INFO: mIOU - All: 5.6121
2021-10-09 08:01:01,752:INFO: ACC - Thing: 34.1646
2021-10-09 08:01:01,752:INFO: mIOU - Thing: 11.0015
2021-10-09 08:01:01,753:INFO: ACC - Stuff: 36.5986
2021-10-09 08:01:01,753:INFO: mIOU - Stuff: 8.6366
2021-10-09 08:01:01,761:INFO: ====== METRIC TEST : cosine ======

2021-10-09 08:01:07,019:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 08:01:07,019:INFO: Batch label size : [128, 320, 320]
2021-10-09 08:01:07,019:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 08:01:07,878:INFO: 0/17
2021-10-09 08:01:24,860:INFO: ACC - All: 24.7779
2021-10-09 08:01:24,860:INFO: mIOU - All: 5.7914
2021-10-09 08:01:24,861:INFO: ACC - Thing: 36.6781
2021-10-09 08:01:24,861:INFO: mIOU - Thing: 9.9772
2021-10-09 08:01:24,861:INFO: ACC - Stuff: 39.3449
2021-10-09 08:01:24,861:INFO: mIOU - Stuff: 9.9257
2021-10-09 08:01:24,871:INFO: ============== Epoch [4] ==============
2021-10-09 08:01:24,871:INFO: Time: [0:1:30:53]
2021-10-09 08:01:24,872:INFO: K-Means loss : 0.06225 | 0.05798
2021-10-09 08:01:24,872:INFO: Training Total Loss : 4.25584
2021-10-09 08:01:24,872:INFO: Training CE Loss (Total | Within | Across) : 4.25584 | 4.23975 | 4.27193
2021-10-09 08:01:24,872:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 08:01:24,872:INFO: [View 1] ACC: 24.4110 | mIoU: 5.6121
2021-10-09 08:01:24,872:INFO: [View 2] ACC: 24.7779 | mIoU: 5.7914
2021-10-09 08:01:24,872:INFO: ========================================

2021-10-09 08:01:26,333:INFO:
============================= [Epoch 5] =============================

2021-10-09 08:01:26,334:INFO: Start computing centroids.
2021-10-09 08:01:46,291:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 08:01:46,292:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 08:01:47,424:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 08:05:53,015:INFO: Initial k-means loss: 0.0522
2021-10-09 08:12:38,567:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0524
2021-10-09 08:20:05,849:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 08:20:05,850:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 08:20:06,614:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 08:23:59,591:INFO: Initial k-means loss: 0.0489
2021-10-09 08:30:36,273:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0488
2021-10-09 08:37:34,446:INFO: -Centroids ready. [Loss: 0.05220| 0.04862/ Time: 0:0:36:8]

2021-10-09 08:37:54,037:INFO: Centroid size : [171, 128]
2021-10-09 08:37:54,038:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 08:37:54,038:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 08:37:57,633:INFO: [Assigning labels] 0 / 194
2021-10-09 08:53:07,770:INFO: Centroid size : [171, 128]
2021-10-09 08:53:07,772:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 08:53:07,772:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 08:53:10,278:INFO: [Assigning labels] 0 / 194
2021-10-09 09:06:18,445:INFO: -Cluster labels ready. [0:0:28:44]

2021-10-09 09:06:18,448:INFO: Start training ...
2021-10-09 09:06:36,385:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 09:06:36,386:INFO: Batch label size : [128, 80, 80]
2021-10-09 09:06:36,386:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 09:06:37,191:INFO: 0 / 388
2021-10-09 09:19:34,159:INFO: 200 / 388
2021-10-09 09:31:35,026:INFO: ====== METRIC TEST : cosine ======

2021-10-09 09:31:40,187:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 09:31:40,187:INFO: Batch label size : [128, 320, 320]
2021-10-09 09:31:40,187:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 09:31:41,007:INFO: 0/17
2021-10-09 09:31:58,495:INFO: ACC - All: 26.0197
2021-10-09 09:31:58,496:INFO: mIOU - All: 5.7386
2021-10-09 09:31:58,497:INFO: ACC - Thing: 38.9953
2021-10-09 09:31:58,497:INFO: mIOU - Thing: 10.7847
2021-10-09 09:31:58,497:INFO: ACC - Stuff: 40.8572
2021-10-09 09:31:58,497:INFO: mIOU - Stuff: 9.4083
2021-10-09 09:31:58,507:INFO: ====== METRIC TEST : cosine ======

2021-10-09 09:32:03,575:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 09:32:03,575:INFO: Batch label size : [128, 320, 320]
2021-10-09 09:32:03,575:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 09:32:04,408:INFO: 0/17
2021-10-09 09:32:21,675:INFO: ACC - All: 26.4511
2021-10-09 09:32:21,675:INFO: mIOU - All: 5.9525
2021-10-09 09:32:21,676:INFO: ACC - Thing: 33.9755
2021-10-09 09:32:21,676:INFO: mIOU - Thing: 10.1442
2021-10-09 09:32:21,676:INFO: ACC - Stuff: 43.5633
2021-10-09 09:32:21,676:INFO: mIOU - Stuff: 10.4873
2021-10-09 09:32:21,685:INFO: ============== Epoch [5] ==============
2021-10-09 09:32:21,685:INFO: Time: [0:1:30:55]
2021-10-09 09:32:21,685:INFO: K-Means loss : 0.05220 | 0.04862
2021-10-09 09:32:21,685:INFO: Training Total Loss : 4.24061
2021-10-09 09:32:21,685:INFO: Training CE Loss (Total | Within | Across) : 4.24061 | 4.22617 | 4.25505
2021-10-09 09:32:21,686:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 09:32:21,686:INFO: [View 1] ACC: 26.0197 | mIoU: 5.7386
2021-10-09 09:32:21,686:INFO: [View 2] ACC: 26.4511 | mIoU: 5.9525
2021-10-09 09:32:21,686:INFO: ========================================

2021-10-09 09:32:23,127:INFO:
============================= [Epoch 6] =============================

2021-10-09 09:32:23,127:INFO: Start computing centroids.
2021-10-09 09:32:43,925:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 09:32:43,926:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 09:32:45,041:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 09:36:46,013:INFO: Initial k-means loss: 0.0465
2021-10-09 09:43:33,686:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0468
2021-10-09 09:50:58,547:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 09:50:58,548:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 09:50:59,286:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 09:54:55,883:INFO: Initial k-means loss: 0.0424
2021-10-09 10:01:33,208:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0429
2021-10-09 10:08:31,983:INFO: -Centroids ready. [Loss: 0.04700| 0.04308/ Time: 0:0:36:8]

2021-10-09 10:08:52,443:INFO: Centroid size : [171, 128]
2021-10-09 10:08:52,444:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 10:08:52,444:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 10:08:55,489:INFO: [Assigning labels] 0 / 194
2021-10-09 10:23:56,728:INFO: Centroid size : [171, 128]
2021-10-09 10:23:56,729:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 10:23:56,729:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 10:23:59,243:INFO: [Assigning labels] 0 / 194
2021-10-09 10:37:08,941:INFO: -Cluster labels ready. [0:0:28:37]

2021-10-09 10:37:08,944:INFO: Start training ...
2021-10-09 10:37:26,120:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 10:37:26,121:INFO: Batch label size : [128, 80, 80]
2021-10-09 10:37:26,121:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 10:37:26,926:INFO: 0 / 388
2021-10-09 10:50:22,936:INFO: 200 / 388
2021-10-09 11:02:26,917:INFO: ====== METRIC TEST : cosine ======

2021-10-09 11:02:32,012:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 11:02:32,012:INFO: Batch label size : [128, 320, 320]
2021-10-09 11:02:32,012:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 11:02:32,855:INFO: 0/17
2021-10-09 11:02:50,435:INFO: ACC - All: 28.0251
2021-10-09 11:02:50,435:INFO: mIOU - All: 5.6963
2021-10-09 11:02:50,436:INFO: ACC - Thing: 46.5006
2021-10-09 11:02:50,436:INFO: mIOU - Thing: 9.2724
2021-10-09 11:02:50,436:INFO: ACC - Stuff: 38.6916
2021-10-09 11:02:50,436:INFO: mIOU - Stuff: 10.6976
2021-10-09 11:02:50,445:INFO: ====== METRIC TEST : cosine ======

2021-10-09 11:02:55,470:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 11:02:55,470:INFO: Batch label size : [128, 320, 320]
2021-10-09 11:02:55,470:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 11:02:56,288:INFO: 0/17
2021-10-09 11:03:13,246:INFO: ACC - All: 27.5272
2021-10-09 11:03:13,247:INFO: mIOU - All: 5.5867
2021-10-09 11:03:13,247:INFO: ACC - Thing: 46.3594
2021-10-09 11:03:13,248:INFO: mIOU - Thing: 9.7279
2021-10-09 11:03:13,248:INFO: ACC - Stuff: 35.9791
2021-10-09 11:03:13,248:INFO: mIOU - Stuff: 9.3963
2021-10-09 11:03:13,257:INFO: ============== Epoch [6] ==============
2021-10-09 11:03:13,257:INFO: Time: [0:1:30:50]
2021-10-09 11:03:13,257:INFO: K-Means loss : 0.04700 | 0.04308
2021-10-09 11:03:13,257:INFO: Training Total Loss : 4.24061
2021-10-09 11:03:13,257:INFO: Training CE Loss (Total | Within | Across) : 4.24061 | 4.22644 | 4.25478
2021-10-09 11:03:13,257:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 11:03:13,257:INFO: [View 1] ACC: 28.0251 | mIoU: 5.6963
2021-10-09 11:03:13,257:INFO: [View 2] ACC: 27.5272 | mIoU: 5.5867
2021-10-09 11:03:13,257:INFO: ========================================

2021-10-09 11:03:14,672:INFO:
============================= [Epoch 7] =============================

2021-10-09 11:03:14,672:INFO: Start computing centroids.
2021-10-09 11:03:34,989:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 11:03:34,990:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 11:03:36,100:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 11:07:44,139:INFO: Initial k-means loss: 0.0410
2021-10-09 11:14:27,744:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0422
2021-10-09 11:22:00,056:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 11:22:00,057:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 11:22:00,802:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 11:25:56,619:INFO: Initial k-means loss: 0.0381
2021-10-09 11:32:32,358:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0392
2021-10-09 11:39:27,901:INFO: -Centroids ready. [Loss: 0.04175| 0.03870/ Time: 0:0:36:13]

2021-10-09 11:39:47,830:INFO: Centroid size : [171, 128]
2021-10-09 11:39:47,830:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 11:39:47,831:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 11:39:51,295:INFO: [Assigning labels] 0 / 194
2021-10-09 11:55:01,698:INFO: Centroid size : [171, 128]
2021-10-09 11:55:01,700:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 11:55:01,700:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 11:55:04,478:INFO: [Assigning labels] 0 / 194
2021-10-09 12:08:17,422:INFO: -Cluster labels ready. [0:0:28:50]

2021-10-09 12:08:17,425:INFO: Start training ...
2021-10-09 12:08:34,990:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 12:08:34,991:INFO: Batch label size : [128, 80, 80]
2021-10-09 12:08:34,991:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 12:08:35,797:INFO: 0 / 388
2021-10-09 12:21:32,956:INFO: 200 / 388
2021-10-09 12:33:28,059:INFO: ====== METRIC TEST : cosine ======

2021-10-09 12:33:33,148:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 12:33:33,149:INFO: Batch label size : [128, 320, 320]
2021-10-09 12:33:33,149:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 12:33:33,965:INFO: 0/17
2021-10-09 12:33:51,319:INFO: ACC - All: 29.1677
2021-10-09 12:33:51,320:INFO: mIOU - All: 5.5887
2021-10-09 12:33:51,320:INFO: ACC - Thing: 48.8678
2021-10-09 12:33:51,321:INFO: mIOU - Thing: 10.6066
2021-10-09 12:33:51,321:INFO: ACC - Stuff: 36.8531
2021-10-09 12:33:51,321:INFO: mIOU - Stuff: 8.3272
2021-10-09 12:33:51,330:INFO: ====== METRIC TEST : cosine ======

2021-10-09 12:33:56,272:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 12:33:56,272:INFO: Batch label size : [128, 320, 320]
2021-10-09 12:33:56,272:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 12:33:57,102:INFO: 0/17
2021-10-09 12:34:14,082:INFO: ACC - All: 29.2309
2021-10-09 12:34:14,083:INFO: mIOU - All: 5.6099
2021-10-09 12:34:14,083:INFO: ACC - Thing: 43.6392
2021-10-09 12:34:14,083:INFO: mIOU - Thing: 10.0625
2021-10-09 12:34:14,083:INFO: ACC - Stuff: 43.5332
2021-10-09 12:34:14,083:INFO: mIOU - Stuff: 9.2222
2021-10-09 12:34:14,092:INFO: ============== Epoch [7] ==============
2021-10-09 12:34:14,092:INFO: Time: [0:1:31:0]
2021-10-09 12:34:14,092:INFO: K-Means loss : 0.04175 | 0.03870
2021-10-09 12:34:14,092:INFO: Training Total Loss : 4.24304
2021-10-09 12:34:14,092:INFO: Training CE Loss (Total | Within | Across) : 4.24304 | 4.22831 | 4.25776
2021-10-09 12:34:14,093:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 12:34:14,093:INFO: [View 1] ACC: 29.1677 | mIoU: 5.5887
2021-10-09 12:34:14,093:INFO: [View 2] ACC: 29.2309 | mIoU: 5.6099
2021-10-09 12:34:14,093:INFO: ========================================

2021-10-09 12:34:15,525:INFO:
============================= [Epoch 8] =============================

2021-10-09 12:34:15,525:INFO: Start computing centroids.
2021-10-09 12:34:35,588:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 12:34:35,588:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 12:34:36,683:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 12:38:39,487:INFO: Initial k-means loss: 0.0380
2021-10-09 12:45:18,945:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0379
2021-10-09 12:52:40,855:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 12:52:40,856:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 12:52:41,605:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 12:56:31,773:INFO: Initial k-means loss: 0.0354
2021-10-09 13:02:55,899:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0355
2021-10-09 13:09:40,064:INFO: -Centroids ready. [Loss: 0.03780| 0.03533/ Time: 0:0:35:25]

2021-10-09 13:10:00,539:INFO: Centroid size : [171, 128]
2021-10-09 13:10:00,540:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 13:10:00,540:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 13:10:03,624:INFO: [Assigning labels] 0 / 194
2021-10-09 13:25:03,904:INFO: Centroid size : [171, 128]
2021-10-09 13:25:03,906:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 13:25:03,906:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 13:25:06,618:INFO: [Assigning labels] 0 / 194
2021-10-09 13:38:03,685:INFO: -Cluster labels ready. [0:0:28:23]

2021-10-09 13:38:03,689:INFO: Start training ...
2021-10-09 13:38:20,982:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 13:38:20,983:INFO: Batch label size : [128, 80, 80]
2021-10-09 13:38:20,983:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 13:38:21,790:INFO: 0 / 388
2021-10-09 13:51:06,870:INFO: 200 / 388
2021-10-09 14:03:00,153:INFO: ====== METRIC TEST : cosine ======

2021-10-09 14:03:05,228:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 14:03:05,229:INFO: Batch label size : [128, 320, 320]
2021-10-09 14:03:05,229:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 14:03:06,059:INFO: 0/17
2021-10-09 14:03:23,198:INFO: ACC - All: 30.1595
2021-10-09 14:03:23,199:INFO: mIOU - All: 5.5250
2021-10-09 14:03:23,199:INFO: ACC - Thing: 50.6786
2021-10-09 14:03:23,199:INFO: mIOU - Thing: 10.2774
2021-10-09 14:03:23,200:INFO: ACC - Stuff: 47.0020
2021-10-09 14:03:23,200:INFO: mIOU - Stuff: 9.5605
2021-10-09 14:03:23,209:INFO: ====== METRIC TEST : cosine ======

2021-10-09 14:03:28,038:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 14:03:28,039:INFO: Batch label size : [128, 320, 320]
2021-10-09 14:03:28,039:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 14:03:28,890:INFO: 0/17
2021-10-09 14:03:45,625:INFO: ACC - All: 29.3897
2021-10-09 14:03:45,626:INFO: mIOU - All: 5.4567
2021-10-09 14:03:45,627:INFO: ACC - Thing: 43.9095
2021-10-09 14:03:45,627:INFO: mIOU - Thing: 10.5183
2021-10-09 14:03:45,627:INFO: ACC - Stuff: 45.5076
2021-10-09 14:03:45,627:INFO: mIOU - Stuff: 9.3557
2021-10-09 14:03:45,636:INFO: ============== Epoch [8] ==============
2021-10-09 14:03:45,636:INFO: Time: [0:1:29:30]
2021-10-09 14:03:45,636:INFO: K-Means loss : 0.03780 | 0.03533
2021-10-09 14:03:45,636:INFO: Training Total Loss : 4.24918
2021-10-09 14:03:45,636:INFO: Training CE Loss (Total | Within | Across) : 4.24918 | 4.23417 | 4.26419
2021-10-09 14:03:45,636:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 14:03:45,636:INFO: [View 1] ACC: 30.1595 | mIoU: 5.5250
2021-10-09 14:03:45,636:INFO: [View 2] ACC: 29.3897 | mIoU: 5.4567
2021-10-09 14:03:45,636:INFO: ========================================

2021-10-09 14:03:47,060:INFO:
============================= [Epoch 9] =============================

2021-10-09 14:03:47,060:INFO: Start computing centroids.
2021-10-09 14:04:07,283:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 14:04:07,284:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 14:04:08,379:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 14:08:09,021:INFO: Initial k-means loss: 0.0350
2021-10-09 14:14:49,329:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0352
2021-10-09 14:22:08,428:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 14:22:08,429:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 14:22:09,189:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 14:26:04,157:INFO: Initial k-means loss: 0.0321
2021-10-09 14:32:28,147:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0322
2021-10-09 14:39:12,409:INFO: -Centroids ready. [Loss: 0.03553| 0.03246/ Time: 0:0:35:25]

2021-10-09 14:39:32,498:INFO: Centroid size : [171, 128]
2021-10-09 14:39:32,499:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 14:39:32,499:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 14:39:35,481:INFO: [Assigning labels] 0 / 194
2021-10-09 14:54:35,884:INFO: Centroid size : [171, 128]
2021-10-09 14:54:35,885:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 14:54:35,885:INFO: Batch feature size : [256, 128, 80, 80]

2021-10-09 14:54:38,596:INFO: [Assigning labels] 0 / 194
2021-10-09 15:07:35,135:INFO: -Cluster labels ready. [0:0:28:23]

2021-10-09 15:07:35,139:INFO: Start training ...
2021-10-09 15:07:51,720:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 15:07:51,721:INFO: Batch label size : [128, 80, 80]
2021-10-09 15:07:51,721:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 15:07:52,526:INFO: 0 / 388
2021-10-09 15:20:38,585:INFO: 200 / 388
2021-10-09 15:32:32,352:INFO: ====== METRIC TEST : cosine ======

2021-10-09 15:32:37,493:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 15:32:37,494:INFO: Batch label size : [128, 320, 320]
2021-10-09 15:32:37,494:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 15:32:38,323:INFO: 0/17
2021-10-09 15:32:55,815:INFO: ACC - All: 29.9956
2021-10-09 15:32:55,816:INFO: mIOU - All: 5.1303
2021-10-09 15:32:55,816:INFO: ACC - Thing: 48.4731
2021-10-09 15:32:55,817:INFO: mIOU - Thing: 9.0006
2021-10-09 15:32:55,817:INFO: ACC - Stuff: 41.6060
2021-10-09 15:32:55,817:INFO: mIOU - Stuff: 8.6583
2021-10-09 15:32:55,826:INFO: ====== METRIC TEST : cosine ======

2021-10-09 15:33:00,872:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 15:33:00,872:INFO: Batch label size : [128, 320, 320]
2021-10-09 15:33:00,872:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 15:33:01,720:INFO: 0/17
2021-10-09 15:33:18,516:INFO: ACC - All: 30.1750
2021-10-09 15:33:18,517:INFO: mIOU - All: 5.0784
2021-10-09 15:33:18,517:INFO: ACC - Thing: 50.6382
2021-10-09 15:33:18,518:INFO: mIOU - Thing: 11.0137
2021-10-09 15:33:18,518:INFO: ACC - Stuff: 38.0510
2021-10-09 15:33:18,518:INFO: mIOU - Stuff: 7.3415
2021-10-09 15:33:18,527:INFO: ============== Epoch [9] ==============
2021-10-09 15:33:18,527:INFO: Time: [0:1:29:31]
2021-10-09 15:33:18,527:INFO: K-Means loss : 0.03553 | 0.03246
2021-10-09 15:33:18,527:INFO: Training Total Loss : 4.25482
2021-10-09 15:33:18,527:INFO: Training CE Loss (Total | Within | Across) : 4.25482 | 4.23967 | 4.26996
2021-10-09 15:33:18,527:INFO: Training MSE Loss (Total) : 0.00000
2021-10-09 15:33:18,527:INFO: [View 1] ACC: 29.9956 | mIoU: 5.1303
2021-10-09 15:33:18,527:INFO: [View 2] ACC: 30.1750 | mIoU: 5.0784
2021-10-09 15:33:18,527:INFO: ========================================

2021-10-09 15:33:19,426:INFO: Start computing centroids.
2021-10-09 15:33:25,274:INFO: Batch input size : [256, 3, 320, 320]
2021-10-09 15:33:25,275:INFO: Batch feature : [256, 128, 80, 80]
2021-10-09 15:33:26,203:INFO: [Saving features]: 0 / 194 | [K-Means Loss]: 0.0000
2021-10-09 15:36:31,440:INFO: Initial k-means loss: 0.0308
2021-10-09 15:42:45,329:INFO: [Saving features]: 100 / 194 | [K-Means Loss]: 0.0308
2021-10-09 15:49:19,710:INFO: -Centroids ready. [Loss: 0.03062/ Time: 0:0:16:0]

2021-10-09 15:49:19,713:INFO: ====== METRIC TEST : cosine ======

2021-10-09 15:49:24,732:INFO: Batch input size : [128, 3, 320, 320]
2021-10-09 15:49:24,732:INFO: Batch label size : [128, 320, 320]
2021-10-09 15:49:24,732:INFO: Batch feature size : [128, 128, 80, 80]

2021-10-09 15:49:25,579:INFO: 0/17
2021-10-09 15:49:41,669:INFO: ACC - All: 34.5429
2021-10-09 15:49:41,669:INFO: mIOU - All: 2.7377
2021-10-09 15:49:41,670:INFO: ACC - Thing: 40.3941
2021-10-09 15:49:41,670:INFO: mIOU - Thing: 4.2091
2021-10-09 15:49:41,670:INFO: ACC - Stuff: 66.9168
2021-10-09 15:49:41,670:INFO: mIOU - Stuff: 4.6067
2021-10-09 15:49:41,679:INFO: Average overall pixel accuracy [NEW] : 34.543 +/- 0.000.
2021-10-09 15:49:41,679:INFO: Average mIoU [NEW] : 2.738 +/- 0.000.
2021-10-09 15:49:41,679:INFO: Experiment done. [0:15:22:59]

I got only 2.7377 mIoU after training. Was this result as expected? Or do you have any suggestions for this full-class setting? Thanks.

Hello,

Thanks for this great work. I have a question about the network in this paper. I saw that you use FPN which is, I think, very similar with U-net. So, why you choose FPN? I think U-net is used more frequent in segmentation.

Thanks,
zihui

Hi Cho:

It's me again:) This time I try to reproduce the result on Cityscapes, which are 65.50 acc and 12.31 mIoU that reported in your paper, by doing the following steps:

1. Download Cityscapes data 'gtFine_trainvaltest.zip' and 'leftImg8bit_trainvaltest.zip' from the official Cityscapes website.

2. Make a projection that change the indices in each 'xxx_gtFine_labelIds.png' by the following dictionary that maps indices from [-1, 33] to [-1, 26]:
   dict = {0: -1, 1: -1, 2: -1, 3: -1, 4: -1, 5: -1, 6: -1, 7: 0, 8: 1, 9: 2, 10: 3, 11: 4, 12: 5, 13: 6, 14: 7, 15: 8, 16: 9, 17: 10, 18: 11, 19: 12, 20: 13, 21: 14, 22: 15, 23: 16, 24: 17, 25: 18, 26: 19, 27: 20, 28: 21, 29: 22, 30: 23, 31: 24, 32: 25, 33: 26, -1: -1}

3. Keep all the rest hyperparameters the same as what they are in 'train_picie.py' and 'train_picie.sh'.

4. Use 2975 training samples for training, and 500 validation samples for evaluation.

However, I cannot get the desired results but only got 7.293 +/- 0.074 and 34.779 +/- 0.350 for mIoU and accuracy, respectively.

I wonder if I took the right way to reproduce the results on Cityscapes, or some additional modifications to the hyperparameters or code are needed to get the right results?

Thanks!

Interesting work!

Can you tell me about the training time? (and the corresponding GPU used for training)

thanks for this work.
anyway, i have my own dataset with .jpg for img and .png for label.
i wonder is there any way to use my own dataset?

Dear authors,

thank you very much for your work. I would be interested in using your model trained on Cityscapes, can you please provide it?
Also, I wonder how exactly do you evaluate your Cityscapes model. In the paper, you write

After filtering out void group, we have 27 categories.

However, in the standard Cityscapes evaluation, only 19 classes are evaluated. Can you please explain how exactly do you do the evaluation?

Thank you very much in advance.

I tested the source code on the coco dataset， But now I want to train own data set，So how to get fine_to_coarse_dict.pickle ？

Thank you very much for such a meaningful work, but I have a question here that I want to ask. In the process of training the cityscapes dataset,the higher the acc indicator, the smaller the miou.Have you ever encountered this situation?

Hi, thank you for your amazing work.
Will codes for over clustering with PiCIE (PiCIE+H) released>

Thanks for your great work!

How can I get the validation annotations here?

Thanks,

Hello, Jang Hyun Cho,

I used your code with my own dataset loaded for training, but I cannot get the stable and good results as yours with COCO-stuff dataset. My own dataset consists of more than 10k raw images (about 1/5 of the COCO-stuff dataset you've used) with only 2 classes ground truth annotation for evaluation. I suspect the model collapse during the mini-batch-kmeans clustering process but I'm not sure and don't know how to solve it. And I wonder if the volume of my own dataset is so small compared to your dataset that leading to bad results, or it is caused by my modification of the original model or the training hyperparameters.

Could you give me a help in detail,
thank you very much,
looking forward to your reply.

Hello,
Thanks for your excellent paper.
There is one thing that confused me.

Your paper applies the same geometric transformation to the input feature and output feature, does this mean that the input and the output feature have the same dimension? Thus we can utilize the totally same transformation.

Hello, thanks for your work and making it public.
In the compute_labels function, after centroid assignments are found, assignment counts are accumulated to get the CE weights due to possible imbalance. However, classes are weighted directly proportional to assignment counts but not inversely proportional. I feel the weights should be inversely related to the pixel counts if the aim is solving the imbalance. Are there any reasons to make it directly proportional?

I tried to run PICIE with the VOC2012 dataset because of its small size, but the results were not good as I expected.
Have you ever tried to run with VOC2012 before? Is there a kind of settings I should focus on? Thank you, it's a good work.

=== The best result ====
ACC - All: 30.4907
mIOU - All: 17.7580

File "/home/workshop/PiCIE/data/coco_eval_dataset.py", line 108, in
fine_to_coarse_map = np.vectorize(lambda x: fine_to_coarse_dict[x]) # not in-place.
KeyError: 183

Traceback (most recent call last):
File "train_picie.py", line 358, in
main(args, logger)
File "train_picie.py", line 190, in main
testset = EvalCOCO(args.data_root, res=args.res, split='val', mode='test', stuff=args.stuff, thing=args.thing)
File "/home/jinglai.whr/code/PiCIE/data/coco_eval_dataset.py", line 40, in init
self.fine_to_coarse = self._get_fine_to_coarse()
File "/home/jinglai.whr/code/PiCIE/data/coco_eval_dataset.py", line 100, in _get_fine_to_coarse
with open(os.path.join(self.root, FINE_TO_COARSE_PATH), "rb") as dict_f:
FileNotFoundError: [Errno 2] No such file or directory: 'datasets/coco/fine_to_coarse_dict.pickle'

Hi, how to get the pickle file "fine_to_coarse_dick.pickle", thanks!

Hello, thanks for sharing your excellent work!

I have a question about your code, especially coco_train_dataset.py.

Is there any issue for this?

Thanks!

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1607370156314/work/aten/src/THC/THCCachingHostAllocator.cpp line=278 error=700 : an illegal memory access was encountered
Traceback (most recent call last):
File "train_picie.py", line 376, in
main(args, logger)
File "train_picie.py", line 271, in main
train_loss, train_cet, cet_within, cet_across, train_mse = train(args, logger, trainloader_loop, model, classifier1, classifier2, criterion1, criterion2, optimizer, epoch)
File "train_picie.py", line 164, in train
loss.backward()
File "/workspace/arijit/miniconda3/envs/picie_x/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/workspace/arijit/miniconda3/envs/picie_x/lib/python3.7/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered

Please help.
Thanks in advance.

Thanks for the interesting research.

Do you have any experimental results at resolutions higher than 320x320?
Does a higher resolution improve the detail of an object?

I tried to decompress the package picie_city.tar, but failed. It looks like a corrupt file.