lars-research / autosf Goto Github PK

Hi there,

I am trying to reproduce the results you have on the ogbl-biokg knowledge graph. You name some tuned hyperparameters in the README and I am a bit confused about that.
Since there are multiple values for each hyperparameter given there, I am not sure which one of those is the one that was found to deliver the best result. Did you do a grid search over them? And if you did, could you possibly share the code? Since right now, I am not able to reproduce the scoring results when I run the examples.sh script.

Thanks in advance!

Best,

Sophia

Hi, I'm trying to reproduce your work, I'm using 4 GeForce RTX with 10G of memory. But after few minutes, the training seems to hang.

 python train.py --task_dir KG_Data/WN18 --optim adagrad --lamb 0.000282 --lr 0.37775 --n_dim 64 --n_epoch 250 --n_batch 1024 --epoch_per_test 50 --test_batch_size 50 --thres 0.0 --parrel 5 --decay_rate 0.99456
B=4 Iter 1      sampled 5 candidate state for evaluate 12929
new: 0 [3 2 0 1] 4
new: 1 [0 2 1 3] 4
new: 2 [2 1 3 0] 4
new: 3 [2 3 0 1] 4
new: 4 [0 1 2 3] 4

nvidia-smi only shows a 1821MB memory used. So i guessed it's not the GPU
How long do you think the training takes?

Thanks

Hi,
I am using two 1080Ti for the training and set the --parrel as 2 (update: even 1 would cause same problem)
However, I get the bellow error although I notice "best_score = 0" in line 94 of train.py, any idea to fix it?
Thanks!

B=4 Iter 1 sampled 5 candidate state for evaluate 12846
newID: 1 newID: 0 [2 3 0 1] 4
[3 2 0 1] 4
newID: 2 [0 1 2 3] 4
newID: 3 [1 3 2 0] 4
newID: 4 [0 2 1 3] 4

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "~/.conda/envs/dgl-ke/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "~/AutoSF/train.py", line 66, in run_model
    best_mrr, best_str = model.train(train_data, tester_val, tester_tst)
  File "~/AutoSF/base_model.py", line 79, in train
    return best_mrr, best_str
UnboundLocalError: local variable 'best_str' referenced before assignment
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "~/AutoSF/train.py", line 155, in <module>
    scor = score.get()
  File "~/.conda/envs/dgl-ke/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
UnboundLocalError: local variable 'best_str' referenced before assignment

Good evening. I have a question , regarding your code for the ogb wikidata. I've been working on graph neural networks lately, but because it's something new to me I'm having difficulties. Could I find somewhere documentation for the code and more specifically for the methods.Thanks in advance.

@yzhangee @quanmingyao @skeletondyh Hi, I'm having a problemp similar to #2

1. After training with bash run.sh, I get the following results saved in WN18_perf.txt file as

1 2 0 3 		best_performance: 0.7187	0.8277		0.7131	0.8213
3 0 1 2 		best_performance: 0.7884	0.8459		0.7877	0.8470
1 0 2 3 		best_performance: 0.9488	0.9533		0.9488	0.9548
2 3 0 1 		best_performance: 0.9486	0.9541		0.9490	0.9565
0 1 2 3 		best_performance: 0.8099	0.9466		0.8128	0.9503
2 3 0 1 2 3 3 1 0 2 2 1 		best_performance: 0.9482	0.9534		0.9483	0.9546
2 3 0 1 2 3 3 -1 2 2 2 1 		best_performance: 0.9454	0.9536		0.9450	0.9542
etc..

I would like to know what these numbers (e.g. 0.9454 0.9536 0.9450 0.9542) represent? Do they represent the hit? I your paper, you reported the MRR, H@1 and the H@10. But here, i see four numbers for each function.

2. It is possible to only test the model using the obtained scoring function f(h,r,t) for evaluation. It will be good if you can add in the read me the command to only test the model by specifying a scoring function

Thanks

please I would like to know how to interpret this result

Hi, I'm very interested in this meaningful work. And I notice that the triplet classification result is shown in the paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding". But it does not seem to exist in this repository.