Why `Tapeable*`? about are-gnn-defenses-robust

[LoadingByte]

In brief, the standard module and optimizer classes are not suited to preserve gradient information about how parameter updates came to be. However, this information is needed to differentiate through the training procedure (which is done by Metattack and related attacks).

To solve that issue, we replace the standard PyTorch classes with out own versions. Notable changes are:

• Tensors wrapped by our TapeableParameter class may have a gradient.
• Our tapeable optimizers avoid inplace operations and the no_grad() context.

Does this help?