sohyun-l / fifo Goto Github PK

Hi,

Thank you for sharing the code of your work.

I followed the training process on NVIDIA RTX A6000 GPU as below:

$ python main.py --file-name 'FIFO_model' --restore-from '.pretrained/Cityscapes_pretrained_model.pth' --restore-from-fogpass './pretrained/FogPassFilter_pretrained.pth' --modeltrain 'train'

And I got result of evaluation on Foggy Zurich, Foggy Driving Dense, and Cityscapes Lindau:

pth name: FIFO2000.pth, Foggy Zurich: 39.56 Foggy Driving Dense: 38.16 Foggy Driving: 47.57 Cityscapes lindau: 67.06
pth name: FIFO4000.pth, Foggy Zurich: 41.91 Foggy Driving Dense: 35.25 Foggy Driving: 47.07 Cityscapes lindau: 68.89
pth name: FIFO6000.pth, Foggy Zurich: 42.66 Foggy Driving Dense: 36.4 Foggy Driving: 46.3 Cityscapes lindau: 68.45
pth name: FIFO8000.pth, Foggy Zurich: 44.93 Foggy Driving Dense: 37.8 Foggy Driving: 47.38 Cityscapes lindau: 60.95
pth name: FIFO10000.pth, Foggy Zurich: 42.11 Foggy Driving Dense: 36.93 Foggy Driving: 47.48 Cityscapes lindau: 67.99
pth name: FIFO12000.pth, Foggy Zurich: 44.97 Foggy Driving Dense: 38.88 Foggy Driving: 48.65 Cityscapes lindau: 63.42
pth name: FIFO14000.pth, Foggy Zurich: 43.95 Foggy Driving Dense: 36.22 Foggy Driving: 46.08 Cityscapes lindau: 68.48
pth name: FIFO16000.pth, Foggy Zurich: 43.45 Foggy Driving Dense: 36.41 Foggy Driving: 46.17 Cityscapes lindau: 72.79
pth name: FIFO18000.pth, Foggy Zurich: 45.52 Foggy Driving Dense: 39.41 Foggy Driving: 47.76 Cityscapes lindau: 67.81
pth name: FIFO20000.pth, Foggy Zurich: 43.78 Foggy Driving Dense: 35.51 Foggy Driving: 46.02 Cityscapes lindau: 67.63
pth name: FIFO25000.pth, Foggy Zurich: 40.43 Foggy Driving Dense: 39.66 Foggy Driving: 47.84 Cityscapes lindau: 61.55
pth name: FIFO30000.pth, Foggy Zurich: 40.05 Foggy Driving Dense: 39.42 Foggy Driving: 46.0 Cityscapes lindau: 67.57
pth name: FIFO35000.pth, Foggy Zurich: 42.91 Foggy Driving Dense: 40.02 Foggy Driving: 46.79 Cityscapes lindau: 66.17
pth name: FIFO40000.pth, Foggy Zurich: 46.28 Foggy Driving Dense: 39.63 Foggy Driving: 47.21 Cityscapes lindau: 66.35
pth name: FIFO45000.pth, Foggy Zurich: 46.30 Foggy Driving Dense: 38.57 Foggy Driving: 45.88 Cityscapes lindau: 68.62
pth name: FIFO50000.pth, Foggy Zurich: 42.95 Foggy Driving Dense: 39.42 Foggy Driving: 46.0 Cityscapes lindau: 68.14
pth name: FIFO55000.pth, Foggy Zurich: 47.23 Foggy Driving Dense: 36.35 Foggy Driving: 45.2 Cityscapes lindau: 66.59
pth name: FIFO60000.pth, Foggy Zurich: 44.85 Foggy Driving Dense: 38.54 Foggy Driving: 45.64 Cityscapes lindau: 68.90
pth name: FIFO65000.pth, Foggy Zurich: 47.59 Foggy Driving Dense: 40.71 Foggy Driving: 46.79 Cityscapes lindau: 66.45
pth name: FIFO70000.pth, Foggy Zurich: 45.73 Foggy Driving Dense: 38.58 Foggy Driving: 46.89 Cityscapes lindau: 61.37
pth name: FIFO75000.pth, Foggy Zurich: 41.55 Foggy Driving Dense: 39.5 Foggy Driving: 45.98 Cityscapes lindau: 69.37
pth name: FIFO80000.pth, Foggy Zurich: 43.80 Foggy Driving Dense: 40.92 Foggy Driving: 47.55 Cityscapes lindau: 68.76
pth name: FIFO85000.pth, Foggy Zurich: 41.53 Foggy Driving Dense: 37.28 Foggy Driving: 45.46 Cityscapes lindau: 60.95
pth name: FIFO90000.pth, Foggy Zurich: 41.64 Foggy Driving Dense: 37.11 Foggy Driving: 44.7 Cityscapes lindau: 65.16
pth name: FIFO95000.pth, Foggy Zurich: 43.69 Foggy Driving Dense: 43.97 Foggy Driving: 48.41 Cityscapes lindau: 64.76

However, the results are not good as reported in paper especially on Foggy Driving Dense dataset.
I am wondering (1) if you have any advice for training and (2) the way you selected the 'FIFO_final_model' ?

Thank you in advance.

Hi.

When I tried to implement your code according to your instruction, I encountered some issues.

In ./configs/train_config.py, DATA_LIST_RF is set to './data/Foggy_Zurich/lists_file_names/RGB_sum_filenames.txt'.

However, when I downloaded the Foggy Zurich dataset, there was no 'lists_file_names/RGB_sum_filenames.txt'.

Did you manually merge 'RGB_medium_filenames.txt' and 'RGB_light_filenames.txt' in the Foggy Zurich dataset into 'RGB_sum_filenames.txt'?

I'm looking forward to hearing your reply.

Thank you!

The Fog Style Matching Loss in the paper：
\mathcal{L}{\mathrm{fsm}}^{l}\left(\mathbf{f}^{a, l}, \mathbf{f}^{b, l}\right)=\frac{1}{4 d{l}^{2} n_{l}^{2}} \sum_{i=1}^{d_{l}}\left(\mathbf{f}{i}^{a, l}-\mathbf{f}{i}^{b, l}\right)^{2}

The code of Fog Style Matching Loss:
In main.py:
... ...
393 fog_factor_b = fogpassfilter(vector_b_gram)
394 fog_factor_a = fogpassfilter(vector_a_gram)
395 half = int(fog_factor_b.shape[0]/2)
397 layer_fsm_loss += fsm_weights[layer]torch.mean((fog_factor_b/(hbwb) - fog_factor_a/(ha*wa))**2)/half/ b_feature.size(0)
399 loss_fsm += layer_fsm_loss / 4

The code is inconsistent with the paper, which one is correct?

Hi authors, thanks for your good work!
How could I obtain the frosty version of Cityscapes? If possible, could you share it with me.
Looking forward to your reply. Thanks.

Hi,

Thanks for sharing the code of this valuable work.

I was using the following command to train the FIFO model.
python3 main.py --file-name 'FIFO_model' --restore-from './Cityscapes_pretrained_model.pth' --restore-from-fogpass './FogPassFilter_pretrained.pth' --modeltrain 'train'

But I found the performance is not that good. I only get the following results.

Evaluation on Foggy Zurich
===> mIoU: 40.99
Evaluation on Foggy Driving Dense
===> mIoU: 37.74
Evaluation on Foggy Driving
===> mIoU: 47.55
Evaluation on Cityscapes lindau 40
===> mIoU: 64.85

Are there any other tricks for training the model or selecting the model?

Best

Thank you for sharing the code of your excellent work.

I have a question about your implementation.

Could you provide a rough estimate of how much time was required for each model to train?

I'm looking forward to hearing your reply.

Thank you.

Hi, Thanks for sharing the code.

I trained the FIFO and reproduced it on FZ, FDD, FD, CW Lindau datasets.

But I'm unable to reproduce on the ACDC dataset. The performance is lower.
(fog : 51.61, rain : 46.34, night : 14.26, snow : 44.3)

I'm wondering if there are any additional changes needed to achieve the reported performance on the ACDC dataset.

Thanks.

Hello, when I read the paper I noticed that in equation 1 after calculating the difference between the cosine distance and m you also need to square the difference. But your code does not square the difference, how do you explain this difference between the code and equation 1 of the paper?