cvir / tcl Goto Github PK

Hi, Would you mind sharing the data list of Jester used in your paper?

What should be done with the original hmdb51 dataset to train. We look forward to hearing from you

Thanks for this great repo!

Whether pre-training weights, such as ImageNet, are used in the self-supervised pre-training stage? And whether ImageNet pre-training weights or other pre-training weights are used if [TCL→Finetuning] is executed without self-supervised pre-training?

I noticed that the models in the paper did three randomized trials when they were trained using different percentages of labeled samples, how were the seed selected in these three trials?

Hi,
Thanks for releasing this nice repo.
I found the links of mini sth-sth v2 dataset files failed, would you mind fixing the links?

Thank you!

Hi,

Thanks for sharing the code. This is a discussion not a code issue.

I'm wondering whether you have tried plug in other backbone instead of TSM with the proposed contrastive learning? I have tried with another network, but it's not quite working well. I guess it's related to how I tune the model, maybe. But good to check with you guys to see whether other models work well in your experiments.

Thank you very much for your time.

underscore to be replaced by dash

opencv-python==4.4.0.42
scikit-learn>=0.24.2

Hi,
Could you provide an example of data inside Run_x or code for generating data inside this folder?
Thank you.

Hi,
Thank you for sharing the code. It seems the tools folder is missing though README shows how to use it for data processing. Could you please add this folder?
Thank you.

Hi,
Thanks for sharing this nice repo : )!
Could you share the command to train and evaluate the TSM ResNet-18/ResNet-50 baselines?

Thank you!

Thanks for your great work! However,I have met a question.You said in your paper in page5 that " Finally, for finetuning with pseudo-labels, we train our model with both labeled and unlabeled videos having pseudo-label confidence more than 0.8 for 50 epochs" So how to calculate the confidence of pseudo-label ? And Is 0.8 a empirical parameter?

In TCL, do you use In_place TSM instead of Residual TSM?

class make_block_temporal(nn.Module):
    def __init__(self, stage, this_segment=3,n_div=8, second_segments=2):
        super(make_block_temporal, self).__init__()
        self.blocks = nn.ModuleList(list(stage.children()))
        self.second_segments = second_segments
        print('=> Processing stage with {} blocks'.format(len(self.blocks)))
        self.temporal_shift = TemporalShift(n_segment=this_segment, n_div=n_div, second_segments= self.second_segments)
#        for i, b in enumerate(self.blocks):
 #           self.blocks[i]= nn.Sequential(b)
    def forward(self,x,unlabeled=False):
        for i, b in enumerate(self.blocks):
            x= self.temporal_shift(x,unlabeled)
            x = self.blocks[i](x)  #just use x, without unlabeled parameters
        return x

class make_blockres_temporal(nn.Module):
    def __init__(self, stage, this_segment=3,n_div=8, n_round=1, second_segments=2):
        super(make_blockres_temporal, self).__init__()
        self.blocks = nn.ModuleList(list(stage.children()))
        self.second_segments = second_segments
        self.n_round = n_round
        print('=> Processing stage with {} blocks'.format(len(self.blocks))) 
        self.temporal_shift = TemporalShift(n_segment=this_segment, n_div=n_div, second_segments=self.second_segments)
   #     for i, b in enumerate(self.blocks):
  #          self.blocks[i]= nn.Sequential(b)
 #           print(self.blocks[i])

    def forward(self,x,unlabeled=False):
        #print("make_block_res_temporal_called")
        for i, b in enumerate(self.blocks):
            #print(x.size())
            if i% self.n_round == 0:
                #print("size of x is {}".format(x.size()))
                x= self.temporal_shift(x,unlabeled)
            x = self.blocks[i](x)  #just use x, without unlabeled parameters
        return x

You change the function 'make_blockres_temporal', and it works as the function 'make_block_temporal'.

Thank you for your perfect work on the paper Semi-Supervised Action Recognition with Temporal Contrastive Learning.

I want to know what is the function of the parameter 'unlabeled'?

def _forward_impl(self, x, unlabeled=False):
        # See note [TorchScript super()]
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        #print("size of x is layer1 is {}".format(x.size()))
        x = self.layer1(x, unlabeled)
        #print("size of x is layer2 is {}".format(x.size()))

        x = self.layer2(x, unlabeled)
        #print("size of x is layer3 is {}".format(x.size()))

        x = self.layer3(x, unlabeled)
        #print("size of x is layer4 is {}".format(x.size()))

        x = self.layer4(x, unlabeled)

        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)

        return x

In class BasicBlock , the forward function just has two parameters but doesn't have the parameter 'unbeled' .So like 'x = self.layer1(x, unlabeled)' will be wrong.

Thanks for this great repo!

I notice in the paper it says there're several stages in the training procedure in section3.3 (Pretraining→TCL →Finetuning). In section 3.2.1 it also mentioned using a supervised model to initialize weights. Do you mind sharing the training command for each stage in order to reproduce the results in the paper?

Thanks

It seems the provided data config files train_videofolder.txt and val_videofolder.txt do not have the same format as mentioned in README, there're only three numbers in each line. Am I missing something?