facebookresearch / invariantriskminimization Goto Github PK

Hi,

I tried to reproduce Fig. 4 from the paper, the noncausal panels for the cases with scrambling (FOS, FES, POS, PES) are empty. The reason is that the noncausal variables are identified by finding zero weights
here, but after scrambling these are removed. I think the errors should be determined by applying the scrambling on the model weights first. Obviously the figure in the paper does contain graphs, could you be so kind to update the code you used? Thank you!

Hi author,

Thanks for your wonderful work. I am confused about the implementation of the squared gradient norm. In the paper, you use X^{e,i} and X^{e,j} to compute the squared gradient norm, where X^{e,i} and X^{e,j} are two random minibatches. However, it seems that the squared gradient norm is computed with the use of X^{e,i}, and X^{e,i} in colored MNIST. Do these two kinds of implementation work in the same way?

Best regards,
Qing LIAN

Hi lopezpaz, IRM is a fantastic work and I am reading and trying to reproduce it. But I meet some questions.

1. When it comes to the colored MINIST experiment. The paper says it divides the MINIST into 3 environments and colors the images to red and green. So Does it mean the test environment also contains digits of the same color in the training set? I mean, suppose that there are digits 9 with green color, and the test set also contains this kind of images?

2. Can IRM solve multiple classification problems? If yes, should I replace binary loss with cross-entropy loss?

I am looking forward to you reply! Thanks.

Hi,

Thanks for the great work! I am trying to reproduce some results and have a question regarding batch implementation of IRM loss. In Section 3.2 and Appendix D, you suggest to use following to do batch implementation:

def compute_penalty(losses, dummy_w):
    g1 = grad(losses[0::2].mean(), dummy_w, create_graph=True)[0] 
    g2 = grad(losses[1::2].mean(), dummy_w, create_graph=True)[0]
    return (g1 * g2).sum()

I am wondering whether we can do following:

def compute_penalty(losses, dummy_w):
    g = grad(losses.mean(), dummy_w, create_graph=True)[0] 
    return (g ** 2).sum()

You mentioned that the former one is "unbiased estimate of the squared gradient norm", but I am not sure why it is the case. If you can provide some explanation, that would be great.

Thank you!

Hi, what does this parameter mean? Turning it off leads to bad test accuracy. It seems like it is converting the colored images into grayscale, which removes any effect of the confounder, so turning it on works well. How does IRM stack on with this?

Hi author,

Thanks for your wonderful work. But I am confused why "obtain the final label y by flipping ~y with probability 0.25". Without this, IRM may get no better than ERM?

Does the mean_accuracy and cross-entropy loss work with multi-class / categorical logits MLP out of the box?

I wonder why w becomes a scalar 1.0 in IRMv1, why not a vector?

I get the following error when running main.py

ImportError: cannot import name 'SEMICP'

It's a simple fix so I can propose a PR separately