limaosen0 / gxn

Graph Cross Network

GXN

Graph Cross Networks with Vertex Infomax Pooling (NeurIPS 2020) paper

We propose a novel graph cross network (GXN) to achieve comprehensive feature learning from multiple scales of a graph. Based on trainable hierarchical representations of a graph, GXN enables the interchange of intermediate features across scales to promote information flow. Two key ingredients of GXN include a novel vertex infomax pooling (VIPool), which creates multiscale graphs in a trainable manner, and a novel feature-crossing layer, enabling feature interchange across scales. The proposed VIPool selects the most informative subset of vertices based on the neural estimation of mutual information between vertex features and neighborhood features. The intuition behind is that a vertex is informative when it can maximally reflect its neighboring information. The proposed feature-crossing layer fuses intermediate features between two scales for mutual enhancement by improving information flow and enriching multiscale features at hidden layers. The cross shape of the feature-crossing layer distinguishes GXN from many other multiscale architectures. Experimental results show that the proposed GXN improves the classification accuracy by 1.96% and 1.15% on average for graph classification and vertex classification, respectively. Based on the same network, the proposed VIPool consistently outperforms other graph-pooling methods.

Here we show the codes of GXN for graph classification as an example.

Module Requirement

• Python 3.6
• Pytorch 1.1
• numpy

Installation

This implementation is under the "lib/" directory, run

make -j4

to compile the necessary c++ files. After compiling the dependent files, you can go to the root directory and run

sh run_GXN.sh [DATANAME] [DATAFOLD] [GPUNUM]

If you want to train and test a model on COLLAB dataset, you should first unzip files, because the original file is large - for convenience, we upload the compressed file directly. Type

cd ./GraphClassificationData/COLLAB
unzip COLLAB.zip

Acknowledgement

Our program is built based on the code of DGCNN and Graph U-Nets. We appreciate their contribution on the technique development and released codes.

gxn's Issues

Low accuracy

Hi:
I use your published code for this paper without any change, why can't I reproduce some dataset result, such as PROTEINS, MUTAG. There occurs overfitting question in training these datasets.

Higher order Rhop is not implemented

Hi,

In equation (2) page 4 https://arxiv.org/pdf/2010.01804.pdf it is mentioned that we can vary R parameter to consider higher order neighboring systems. However, in the code l didn't find where do you use Rhop parametrer. The following functions are parameterized with R which is not used. How can l incorporate R ? I have missed something ?

Thank you for your consideration and for this nice work !

In the function IndexSelect()

GXN/ops.py

Line 249 in cb01e49

class IndexSelect(nn.Module):

and in the Discriminator()

GXN/ops.py

Line 120 in cb01e49

class Discriminator(nn.Module):

Does your model have true test dataset?

I think your code actually reports the validation result right? More specifically, in here

for epoch in range(cmd_args.num_epochs):
        random.shuffle(train_idxes)
        classifier.train()
        avg_loss = loop_dataset(train_graphs, classifier, mi_loss, train_idxes, epoch, optimizer=optimizer, device=device)
        avg_loss[4] = 0.0
        print('\033[92maverage training of epoch %d: clsloss: %.5f miloss: %.5f loss %.5f acc %.5f auc %.5f\033[0m'
              % (epoch, avg_loss[0], avg_loss[1], avg_loss[2], avg_loss[3], avg_loss[4])) # noqa

        classifier.eval()
        test_loss = loop_dataset(test_graphs, classifier, mi_loss, list(range(len(test_graphs))), epoch, device=device)
        test_loss[4] = 0.0
        print('\033[93maverage test of epoch %d: clsloss: %.5f miloss: %.5f loss %.5f acc %.5f auc %.5f\033[0m'
              % (epoch, test_loss[0], test_loss[1], test_loss[2], test_loss[3], test_loss[4])) # noqa

        with open(logfile, 'a+') as log:
            log.write('test of epoch %d: clsloss: %.5f miloss: %.5f loss %.5f acc %.5f auc %.5f'
                      % (epoch, test_loss[0], test_loss[1], test_loss[2], test_loss[3], test_loss[4]) + '\n')

        if test_loss[3] > max_acc:
            max_acc = test_loss[3]
            fname = './checkpoint_%s/time_%s/FOLD%s/model_epoch%s.pt' % (cmd_args.data, first_timstr, foldidx, str(epoch))
            torch.save(classifier.state_dict(), fname)

with open('./result_%s/result_%s/acc_result_%s_%s.txt' % (cmd_args.data, first_timstr, cmd_args.data, first_timstr), 'a+') as f:
        f.write('\n')
        f.write('Fold index: ' + str(foldidx) + '\t')
        f.write(str(max_acc) + '\n')

Problem in loss computation

Hi, in your paper you say 'In the loss function L, α decays from 2 to 0 during training, where the VIPool needs fast convergence for vertex selection'. However, in your code, the α actually decays from 2 to 1, see here.

Problem in class IndexSelect

Hi, your GXN work is interesting and I'm trying to implement it using DGL. However, I find something strange in your code.
Specifically, in your paper, the criterion function for VIPool involves two function values: T_w(\mathbf{x}_v, \mathbf{y}_{\mathcal{N}_v}) and T_w(\mathbf{x}_v, \mathbf{y}_{\mathcal{N}_u}), where T_w(\mathbf{x}_v, \mathbf{y}_{\mathcal{N}_u}) = \mathcal{S}_w(\mathcal{E}_w(\mathbf{x}_v), \mathcal{P}_w(\mathbf{y}_{\mathcal{N}_u})). And as described in your paper, \mathcal{E} is MLP, \mathcal{P} is some Message-Passing layer.

However, in your code, it seems like you implement these in the form of T_w(\mathbf{y}_{\mathcal{N}_v}, \mathbf{x}_v) and T_w(\mathbf{y}_{\mathcal{N}_u}, \mathbf{x}_v). More specifically, in the class IndexSelect of your code:

class IndexSelect(nn.Module):

    def __init__(self, k, n_h, act,  R=1):
        super().__init__()
        self.k = k
        self.R = R
        self.sigm = nn.Sigmoid()
        self.fc = MLP(n_h, n_h, act)
        self.disc = Discriminator(n_h)
        self.gcn1 = GCN(n_h, n_h)

    def forward(self, seq1, seq2, A, samp_bias1=None, samp_bias2=None):
        h_1 = self.fc(seq1)
        h_2 = self.fc(seq2)
        h_n1 = self.gcn1(A, h_1)

        X = self.sigm(h_n1)
        ret, ret_true = self.disc(X, h_1, h_2, samp_bias1, samp_bias2)
        scores = self.sigm(ret_true).squeeze()
        num_nodes = A.shape[0]
        values, idx = torch.topk(scores, int(num_nodes))
        values1, idx1 = values[:int(self.k*num_nodes)], idx[:int(self.k*num_nodes)]
        values0, idx0 = values[int(self.k*num_nodes):], idx[int(self.k*num_nodes):]

        return ret, values1, idx1, idx0, h_n1

Looks like only X is the output of GCN, while h_1 and h_2 (In my understanding, they represent for \mathcal{P}_w(\mathbf{y}_{\mathcal{N}_v}) and \mathcal{P}_w(\mathbf{y}_{\mathcal{N}_u}) respectively) are output of MLP. If we follow the setting in your paper, shouldn't h_1 and h_2 be the output of GCN?

Baseline

Hi,

Your work is really impressive! I have a question about baseline methods. Do you implement them by yourself or use the public code? To my knowledge, different baseline methods use different GCN module. If you implement them by yourself, could you please share your implementation? Thanks!

Codes for vertex classification

Hi,
Your work is very impressive! Could you please provide the codes of GXN for vertex classification? Thanks!