GithubHelp home page GithubHelp logo

njuvision / awnet Goto Github PK

View Code? Open in Web Editor NEW
17.0 17.0 6.0 6.5 MB

A Dual Camera System for High Spatiotemporal Resolution Video Acquisition (TPAMI 2020)

Home Page: https://NJUVISION.github.io/AWnet

License: MIT License

Python 43.33% Shell 0.37% C++ 9.22% Cuda 29.64% HTML 17.44%

awnet's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

awnet's Issues

Non differentiable Warping

Hi
I am playing around with your code and I learnt that these lines are non-differentiable and do not let the gradient compute. Can you please guide me on how you solved that? or its just me who is having this issue.

    mask = torch.autograd.Variable(torch.ones(x.size())).cuda()
    mask = nn.functional.grid_sample(mask, vgrid)

   # if W==128:
        # np.save('mask.npy', mask.cpu().data.numpy())
        # np.save('warp.npy', output.cpu().data.numpy())

    mask[mask<0.9999] = 0
    mask[mask>0] = 1

    return output*mask,mask

for the simplest code to reproduce the error,

import torch
use_cuda = torch.cuda.is_available()
device = torch.device("cuda" if use_cuda else "cpu")
torch.cuda.set_device(0)
criteria = torch.nn.L1Loss()


# import awnet_pwcnet #please adjust as per your code

net = awnet_pwcnet.PWCDCNet().cuda()
#net = torch.nn.Conv2d(6, 3, 1).cuda()
optimizer = torch.optim.Adam(net.parameters(), lr = 0.1)

inp = Variable(torch.rand(1,3,256,448).to(device),requires_grad = True)
snd = Variable(torch.rand(1,3,256,448).to(device),requires_grad = True)
gt = Variable(torch.rand(1,2,256,448).to(device),requires_grad = True)

x = torch.cat((inp,snd),1)

# for layer in net.children():
#     for param in layer.parameters():
#         print(param)

for i in range(1,5):
    optimizer.zero_grad()
    out = net(x)
    out = F.interpolate(out,scale_factor=4,mode='bilinear',align_corners=False)*20
    loss = criteria(out,gt)
    print(out.shape)
    loss.backward()#retain_graph= True)
    print(net.state_dict())
    optimizer.step()

and i get this error which is caused by these lines as when I replaced the warping function with this one, it started to train.
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Thanks alot.

Pre-trained Weights

Hi,

Its an amazing work. please upload the pre-trained weights to play around with. Thanks.

About the pretrained pwc-net

Hello, in the paper you mentioned using pretrained PWC_Net to initialize our FlowNet, I want to ask how to get this pretrained PWC_Net? Thank you!

Evaluation toolkit

Hi,

Can you please guide me which toolkit you used for the calculation of evaluation metrics? SSIM/PSNR?

Thanks a lot.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.