Question about your sparse implementation about gat

[PetarV-]

Hi Minjie,

Thank you for the kind words and your interest in GAT!

I believe what you are probably missing is adding the identity matrix to the adjacency. The raw sparse matrix, as extracted by Thomas Kipf's code, does not have ones on the diagonal (they only add it in post-processing).

Disclaimer: We have played around with the sparse layer only for the purposes of Pubmed, so cannot guarantee exact reproducibility compared to the dense layer. However, I just now executed the sparse model on Cora once and managed to get 83.7% on the test set.

P.S. If you manage to get a good-looking version of the sparse code running, could you consider submitting a pull request with a models/sp_gat.py and sp_execute_cora.py? All the versions I have implemented seem too hacky for inclusion in this repository. :)

Thanks,
Petar

[jermainewang]

Hi Petar,

Thank you for your reply! I figured out what I missed. When I fed the sparse adj matrix to tensorflow, I incorrectly packed the row index and column index. Basically, I used following codes to convert the adj matrix to tf.SparseTensor:

def process_adj_sp(adj):
  num_nodes = adj.shape[0]
  adj = adj + sp.eye(num_nodes)  # self-loop
  adj[adj > 0.0] = 1.0
  if not sp.isspmatrix_coo(adj):
    adj = adj.tocoo()
  adj = adj.astype(np.float32)
  indices = np.vstack((adj.col, adj.row)).transpose()  # This is where I made a mistake, I used (adj.row, adj.col) instead
  return tf.SparseTensor(indices=indices, values=adj.data, dense_shape=adj.shape)

It worked perfectly now. I think you implementation looks fine for me. I did make some changes, but the main sp_attn_head function followed your implementation. I could try make a pull request later.

Best regards,
Minjie

[KnightOfTheMoonlight]

Hi, @jermainewang and @PetarV- , I just have a quick simple question. Why would you suggest a sparse adjacency bias matrix？ To deal with what scenarios? I tried, it seems not to work for speedup.