megvii-research / tlc Goto Github PK

Hello and thank you for you work.

My input image is 1360x1814 and I get the following error:

 File "/Users/user3/Documents/projects/TLC/basicsr/models/archs/mprnet_arch.py", line 192, in forward
    x = x + y
        ~~^~~
RuntimeError: The size of tensor a (452) must match the size of tensor b (453) at non-singleton dimension 2

tensor's a shape torch.Size([1, 144, 452, 680])
tensor's b shape torch.Size([1, 144, 453, 680])

That happens on some i'th forward step. it is quite strange, that the mismatched happens, isnt it ?

Traceback (most recent call last):
File "basicsr/demo.py", line 11, in
from basicsr.train import parse_options
ModuleNotFoundError: No module named 'basicsr.train'

Some functions are missing from the repo. Could you please check and commit all the files

thankyou

Hi, thank you very much for your exciting research.
Your paper "4.3 Extensibility and Complexity" confirms that adding TLC to UNet, including SE, significantly improves performance with a slight increase in computational complexity. I have a question on this point.

In SE without TLC, the feature map size is (C, 1, 1) after global pooling. After that, it changes to (C/r, 1, 1) and (C, 1, 1) at Fully Connected layers.
The MACs are (2C^2)/r.
With TLC, the feature map is re-weighted by the element-wise attention, so the feature map size after global pooling is (C, H, W). Therefore, the computational complexity, including the Fully Connected layer, is HW*(2C^2)/r.

Is this correct?
If so, I think the computation has increased. Do I misunderstand something?

Thank you in advance for your response.

Hi,
When I click on the pretrained model url link, it gives 404 error and can't be reach so I can not download your checkpoint. Can you load again please? maybe with google drive link.

作者你好，我很佩服你们的工作。
我在看你的代码时发现，在将TLC扩展到SCA层时，在class AvgPool2d中你采用子矩阵和的方法，用kernel_sizekernel_size大小的框（例如384384）遍历特征图（例如1280*720）求出了每一个区域的均值（这些均值是密集的），然后用replicate方法填充到与特征图同等大小获得out,然后将out与特征图点乘（对应元素相乘）。我觉得这样做违背了CA(通道注意力）的初衷，我感觉这样做甚至破坏了原始的特征图了，但是效果又是非常好的，您可以解释以下这其中有什么道理吗，很感谢您的热心解答。

Functions
'create_lmdb_for_gopro',
'create_lmdb_for_rain13k'

Are missing from utils.create_lmdb.py

试了下预测的demo，一张图片大概要3~4秒，请问是正常的么？

在转换tensorRT时候汇报Pad错误，请问这个支持转成tensorRT吗

Hello, I had a lot of questions when I applied your work to the super-resolution model, such as Local_Base and so on. If possible, could you give a concrete example to show us how to apply it?

作者你好，我在加载Restormer的预训练权重文件motion_deblurring.pth时，发生如下错误
/home/guest/anaconda3/envs/daizhiwei/bin/python /usr/daizhiwei/TLC-main/basicsr/demo.py -opt ../options/demo/demo.yml
Disable distributed.
Traceback (most recent call last):
File "/usr/daizhiwei/TLC-main/basicsr/demo.py", line 47, in
main(root_path)
File "/usr/daizhiwei/TLC-main/basicsr/demo.py", line 40, in main
model = create_model(opt)
File "/usr/daizhiwei/TLC-main/basicsr/models/init.py", line 44, in create_model
model = model_cls(opt)
File "/usr/daizhiwei/TLC-main/basicsr/models/image_restoration_model.py", line 68, in init
self.load_network(self.net_g, load_path,
File "/usr/daizhiwei/TLC-main/basicsr/models/base_model.py", line 279, in load_network
load_net = load_net[param_key]
KeyError: 'state_dict'

但是当我加载HINet-GoPro.pth权重和MPRNet-GoPro.pth却没有发生错误，程序能正常运行，请问这与什么有关呢，该如何解决这个问题？

Thank you for your awesome code!

I am hoping you might open-source the log files you have from training. Maybe the training and validation loss as a function of epoch
(and/or batch) with an estimate of the runtime?

Hi, I'm trying to run RestormerLocal using your source code.

What is the meaning of the following code?

self.kernel_size = list(self.base_size)
self.kernel_size[0] = x.shape[2] * self.base_size[0] // train_size[-2]
self.kernel_size[1] = x.shape[3] * self.base_size[1] // train_size[-1]

The kernel size is updated by the 2nd and 3rd rows. Under the default base_size (1.5*train_size) setting, kernel_size becomes larger than the resolution of the input image. So it would be the same with the original Restormer, not TLC version.

https://github.com/megvii-research/tlsc/blob/c5c9bac1557d2f95bf281152f09bda61db578a93/basicsr/models/archs/local_arch.py#L100
It will output negative numbers. That is the reason for 'NaN' for torch.sqrt().

I have meet the following problems, how should I solve them? thanks!
File "/usr/local/lib/python3.7/dist-packages/basicsr/utils/options.py", line 141, in parse_options
for phase, dataset in opt['datasets'].items():
KeyError: 'datasets'

作者你好！
在/TLC-main/basicsr/models/archs/local_arch.py文件中的第31行：
return 'kernel_size={}, base_size={}, stride={}, fast_imp={}'.format(
self.kernel_size, self.base_size, self.kernel_size, self.fast_imp
)
我了解到kernel_size是框的大小，并且框每次移动一个像素距离。
但是这里的stride参数有什么实际意义吗？

Hi @achusky.

I am running this line:
'python basicsr/test.py -opt options/test/GoPro/MPRNetLocal-GoPro.yml'
which gives the following error:
'Traceback (most recent call last):
File "basicsr/test.py", line 11, in
from basicsr.data import create_dataloader, create_dataset
File "/home/misak/Desktop/Projects/tlsc_test/tlsc/basicsr/data/init.py", line 17, in
from basicsr.utils import get_root_logger, scandir
File "/home/misak/Desktop/Projects/tlsc_test/tlsc/basicsr/utils/init.py", line 13, in
from .create_lmdb import (create_lmdb_for_reds, create_lmdb_for_gopro, create_lmdb_for_rain13k)
ImportError: cannot import name 'create_lmdb_for_gopro' from 'basicsr.utils.create_lmdb' (/home/misak/Desktop/Projects/tlsc_test/tlsc/basicsr/utils/create_lmdb.py)'.

Then, I comment the 'create_lmdb_for_gopro, create_lmdb_for_rain13k' from 'basicsr/utils/init.py', line 13, and get the next error:
'Traceback (most recent call last):
File "basicsr/test.py", line 13, in
from basicsr.train import parse_options
ModuleNotFoundError: No module named 'basicsr.train''

There is no basicsr.train module in this project to import parse_options.

Could you, please, try to look on this?

I have done the installation provided in the github page, downloaded the GoPro dataset and pre-trained model.

Thanks,
Misak

Thank you for your sharing.
If I have a trained dehazing network ready, How can I incorporate TLC into my test code

Thanks for your enjoy！

I have a try in Rain100L dataset of my own trained net, but when calculating PSNR and SSIM, it gives me " AssertionError: Image shapes are differnet: (80, 120, 3), (320, 480, 3)". The size of my train test is (320,480).

Look for your reply,thank you!

Thank you for sharing your research!
Will you be able to provide training code so that we can train it on a custom dataset?

Hello, your innovative work is great and has contributed very much to the field of image restoration. I am working in the field of super-resolution, and the super-resolution field also crops the image into some patches during training, for example, the DIV2K dataset is cropped to 192*192 size, but the LR image of the test set is usually smaller than the HR image after cropping, which is equivalent to the training patches are larger than the whole image I will use for testing, is your work also positive and effective in this case? ?