pranoyr / cnn-lstm

CNN LSTM architecture implemented in Pytorch for Video Classification

Hi there 👋

Just working on some Open Source Projects regarding Attention models.

cnn-lstm's Issues

there is a bug about train_loss

when training, the avg_loss = train_loss / log_interval, while train_loss is always 0

Accuracy drop

Thanks for your great work!
I followed the step and running the code successfully.
Unfortunately, the learning curve image shown below show that the accuracy drop to 60% validation accuracy,
where there is a 30% gap with your learning curve.
I don't understand why there are only four label in your tensorboard visualization.
All in all, I'm wondering what's the performance you get while training for 101 labels of UCF101,
and make sure I got anything wrong. Thanks

do you have any testing code that could provide?

change

when l set --sample_duration 32 , it will have below error, do you know how to solve ?

results have some problems

hello, I do it according to readme,the code is running well,but the results is always 50%. (in fact ,the video in the data are same, I replace the one video. now the video are different,but the results is still 50%)，please help me,thank you

About ”with torch.no_grad()“

Hi,
I saw the line with torch.no_grad() in the cnnlstm.py file, this means that the weight of resnet will not be affected by backward. If I delete with torch.no_grad(), the backward will update the weights of cnn and lstm at the same time?

cnnlstm: loss does not drop?

Hi,
The following code is fine-tuned according to the code you wrote. When I only use resnet for training, the loss can drop normally, but when I use resnet+lstm, the loss has been around 0.6 and does not drop. Could you please guide me?

class CNNLSTM(nn.Module):
 def __init__(self, num_classes=2):
     super(CNNLSTM, self).__init__()
     self.resnet = resnet34(pretrained=True)
     self.resnet.fc = nn.Sequential(nn.Linear(self.resnet.fc.in_features, 300))  
     self.lstm = nn.LSTM(input_size=300, hidden_size=256, num_layers=3)
     self.fc1 = nn.Linear(256, 128) # Fully connected layer
     self.fc2 = nn.Linear(128, num_classes) # Fully connected layer
 
 def forward(self, x_3d):
     hidden = None
     x_3d = x_3d.unsqueeze(0) # add
     x_ = list()
     for t in range(x_3d.size(1)):
         # with torch.no_grad():
         x = self.resnet(x_3d[:, t, :, :, :]) # x_3d[:, t, :, :, :].shape: [1,3,224,224] 
         out, hidden = self.lstm(x.unsqueeze(0), hidden)    
         
         x = self.fc1(out[:, -1, :])
         x = F.relu(x)
         x = self.fc2(x)
         if t==0:
             x_ = x
         else:
             x_ = torch.cat([x_, x], dim=0)
     return x_