twtygqyy / pytorch-lapsrn Goto Github PK

I can't seem to reproduce the Set5 (x4) result reported, using the default training options with --cuda. Do you have any idea?

Thanks for your code. I want to know how to get the Set5 (x4) result reported. What's your training datasets in the generate_train_lap_pry file? I can't reproduce it. Thanks!

I used the modified code below to generate the ouput image of the model, but got images filled by noise. I checked the code a lot of time but didn't find any problem. So can u help me?

def PSNR(pred, gt, shave_border=0):
    height, width = pred.shape[:2]
    pred = pred[shave_border:height - shave_border, shave_border:width - shave_border]
    gt = gt[shave_border:height - shave_border, shave_border:width - shave_border]
    imdff = pred - gt
    rmse = math.sqrt(np.mean(imdff ** 2))
    if rmse == 0:
        return 100
    return 20 * math.log10(255.0 / rmse)

def colorize(y, ycbcr):
    img = np.zeros((y.shape[0], y.shape[1], 3))
    img[:,:,0] = y
    img[:,:,1] = ycbcr[:,:,1]
    img[:,:,2] = ycbcr[:,:,2]
    img = Image.fromarray(img, "YCbCr").convert("RGB")
    return img

for image_name in image_list:
    print("Processing ", image_name)
    im_gt_y = sio.loadmat(image_name)['im_gt_y']
    im_b_y = sio.loadmat(image_name)['im_b_y']
    im_l_y = sio.loadmat(image_name)['im_l_y']
    im_b_ycbcr = sio.loadmat(image_name)['im_b_ycbcr']

    im_b_ycbcr = im_b_ycbcr*255.
    im_b_ycbcr[im_b_ycbcr<0] = 0
    im_b_ycbcr[im_b_ycbcr>255.] = 255.

    im_gt_y = im_gt_y.astype(float)
    im_b_y = im_b_y.astype(float)
    im_l_y = im_l_y.astype(float)

    psnr_bicubic = PSNR(im_gt_y, im_b_y,shave_border=opt.scale)
    avg_psnr_bicubic += psnr_bicubic

    im_input = im_l_y/255.

    im_input = Variable(torch.from_numpy(im_input).float()).view(1, -1, im_input.shape[0], im_input.shape[1])

    if cuda:
        model = model.cuda()
        im_input = im_input.cuda()
    else:
        model = model.cpu()

   start_time = time.time()
    HR_2x, HR_4x = model(im_input)
    elapsed_time = time.time() - start_time
    avg_elapsed_time += elapsed_time

    HR_4x = HR_4x.cpu()

    im_h_y = HR_4x.data[0].numpy().astype(np.float32)

    im_h_y = im_h_y*255.
    im_h_y[im_h_y<0] = 0
    im_h_y[im_h_y>255.] = 255.
    im_h_y = im_h_y[0,:,:]


    psnr_predicted = PSNR(im_gt_y, im_h_y,shave_border=opt.scale)
    avg_psnr_predicted += psnr_predicted

    im_h = colorize(im_h_y, im_b_ycbcr)

    im_h.save("store path") ==========>   save the np array as array

I also can't reproduce the Set5 (x4) result reported, using the default training options with --cuda.

I tried to reproduce LAPSRN just same as paper in CVPR, so I changed optim to SGD. Then, it makes loss go inf. So, I changed loss term divided by batchsize as paper mentioned, but still loss go inf. Do you have any idea?

I don't have environment good enoght to generate data with MATLAB.
could you provide the whole patches on T91 and BSDS200 datasets in format .hi5 file generated from 'generate_train_lap_pry.m' ?
thank you

Hi,
Have you tried shared weight between different pyramid levels（as the LapSRN paper said）, the code of you uploaded now seems doesn't do it.

Hello,
I can't replicate your PSNR value using bicubic on Set14 and Set5. I'm having 2db lower. Could share please the images that you used ?

Hi, thanks for your sharing.
In the generate_train_lap_pry.m , label_x2 were got by 'bicubic' on the scale of 1/2 , in the training, loss_x2 was computed by HR_2x and label_x2, shouldn't this label_x2 be ground truth, not bicubic interpolation image?

All I see are x2 x4 x8 using this model. Do you have an idea of how to upscale by x3 ?

Hi, thanks for your sharing.

I can't reproduce the result you reported when I train the model with the default options. I generated the training file with the script you provided. Finally, the PSNR of Set5 is 30.32 which is 0.33 smaller than the result you reported.

I set the random seed to a fixed value. Would this affect the performance? I want to know what's the value of random seed in your model.

Best~~

In the test section, can I achieve one effect: input low resolution test image, input high resolution image, if so, how can I modify the code

how to get 2× psnr from 4× model?

Can you give any list of particular preprocessing needed or in what format should we save our data in hd5 format for training my own custom data. I am currently involved in a research project so we need to benchmark existing methods on our data so getting the hands dirty in implementation is not something i am right now keen in. Any help ?

Thanks for the open source.Now I want to try a super resolution on a tiny image ,so can you show how to train LapSRN x8 model?

usage: demo.py [-h] [--cuda] [--model MODEL] [--image IMAGE] [--scale SCALE]

PyTorch LapSRN Demo

We are experiencing the following attempting to train on some astrophotography images. I have created 128x128 patches of moon photos for our data set, and then used your generate Matlab routine to generate an HDF5 file.

We are attempting to train with: python main_lapsrn.py --nEpochs=50 --threads=1

Software versions do not appear to be specified, so we are using the following library versions in our conda environment:
Python: 3.8.5
PyTorch: 1.7.1
NumPy: 1.19.2
h5py: 2.10.0

We are experiencing the same error as well with SRDenseNet.

Epoch=1, lr=0.0001
Traceback (most recent call last):
File "main_lapsrn.py", line 140, in
main()
File "main_lapsrn.py", line 83, in main
train(training_data_loader, optimizer, model, criterion, epoch)
File "main_lapsrn.py", line 126, in train
print("===> Epoch{}: Loss: {:.10f}".format(epoch, iteration, len(training_data_loader), loss.data[0]))
IndexError: invalid index of a 0-dim tensor. Use tensor.item() in Python or tensor.item<T>() in C++ to convert a 0-dim tensor to a number

I want to train the lapsrn_wgan, can you give me some reference material?

Traceback (most recent call last):
File "main_lapsrn.py", line 140, in
main()
File "main_lapsrn.py", line 83, in main
train(training_data_loader, optimizer, model, criterion, epoch)
File "main_lapsrn.py", line 119, in train
loss_x2.backward(retain_graph=True)
TypeError: backward() got an unexpected keyword argument 'retain_graph'

pytorch version 0.3.1

why all convnet use bias=false even that BN is not used ?

Hi!

I am preparing my own data for training.
I am running the "generate_train.m" script from pytorch-vdsr and I have changed the path to the folder with images (approx 70k images). I have 2 doubts:

1. My images are of different dimensions, so will that cause a problem in training?
2. How do I add labels 'label_x2' and 'label_x4'?

I am to new to HDF format and hence the issue.

'Conv2d' object has no attribute 'padding_mode'

Hi, The code cannot take a random .jpg file as input and only a .mat file.
Is there any way to test the model on a random image?

Hi ,
I would like to ask you about the training time period and the Specifications of the machine that your train your model in ?

hello~ i have a question
one branch aims to extract gray-level features(channel=1)，add to unsample input (channel=3)
but why the result is also gray image？

Thanks for your code, it helps me a lot. But I have some questions about data augmentation. In the generate_train_lap_pry.m, you only used downsizing to make more training data. While in the paper, the author augments the training data in three ways, scaling, rotation and flipping. Your performance is better than the paper, but your training data only has 7488 examples. I'm confused about it.

Thank you for sharing the code.However, according to your project, my PSNR on set5 is only 31.48 instead of 31.65. whats the problem. The dataset I used is T91, BSDS200.

Thanks for your repository which helps my learn the model . However , I have some questions about the code are as following:

 features = features[:, :, :-1, :-1]
 rescaled_img = self.scale_img(rescaled_img)
 rescaled_img = rescaled_img[:, :, 1:-1, 1:-1]

why the start index of slice of features is from 0, and the rescaled_image is from 1?
And why should cut the slice, can we add them directly.

Hi,
We are trying to train and test the LapSRN pytorch code provided by you. I would like to if there is any standard implementation of SSIM. Also its not clear how to give the two inputs to SSIM implementation from skimage. Should it be in uint8 (0 to 255)?

Thank You.

Hello. Please help me how to modify your code LapSRN_WGAN for same size image processing without upsampling input image? I want use LapSRN_WGAN for image processing with same size

Should I modify this line of code in https://github.com/twtygqyy/pytorch-LapSRN/blob/master/lapsrn_wgan.py

Change " nn.ConvTranspose2d(in_channels=64, out_channels=64, kernel_size=4, stride=2, padding=1, bias=False)", to nn.Conv2d(in_channels=64, out_channels=64, kernel_size=4, stride=1, padding=1, bias=False),

Or may be I can just remove this layer?

can you please help me know where the results are being stored in which folder