mccreezhao / qamface Goto Github PK

I haven't seen your paper yet but was just wondering how it was determined to use an initial LR of 0.2 instead of the typical 0.1.
While training IR_152 backend, I saw training loss jump significantly to 100.
Also, have you tried retraining using a pretrained backbone?

the loss function used in the project was loss = self.focalLoss(thetas, labels)??? not Additive Angular Margin Loss?

I notice you only went to epoch 22, which is still stage 1. Are you still training this?

I noticed there's no code to set which GPUs are to be used. There's also no synchronisation code in the loss function for multiple GPUs.

How did you train it on 4 GPUs without this, or is it no longer needed in modern PyTorch?

I modified your QAMFace to include the support:

        if self.device_id == None:
            kernel_norm = l2_norm(self.weight, axis=0)
            cos_theta = torch.mm(embbedings, kernel_norm)
        else:
            x = embbedings
            sub_kernels = torch.chunk(self.weight, len(self.device_id), dim=1)
            temp_x = x.cuda(self.device_id[0])
            kernel_norm = l2_norm(sub_kernels[0], axis = 0).cuda(self.device_id[0])
            cos_theta = torch.mm(temp_x, kernel_norm)
            for i in range(1, len(self.device_id)):
                temp_x = x.cuda(self.device_id[i])
                kernel_norm = l2_norm(sub_kernels[i], axis = 0).cuda(self.device_id[i])
                cos_theta = torch.cat((cos_theta, torch.mm(temp_x, kernel_norm).cuda(self.device_id[0])), dim=1)

Ideally the input data is normalised between -1 and 1 because fp32 and fp16 have better precision in that range.
Is there a reason why it is commented out in the code?

The batch size in config.py is not used for anything (instead it used the -b option).
This confused me initially. Maybe it could change the default value.

Hello and thanks for your work!

Did you try to use cross-entropy loss with your QAAM head? If yes, what were the results compared to the focal-loss?