csjtx1021 / scalable-and-parallel-dgbo

This code is implemented according to paper "Scalable and Parallel Deep Bayesian Optimization on Attributed Graphs", accepted by TNNLS. (Python2/TensorFlow)

This code is implemented according to paper "Scalable and Parallel Deep Bayesian Optimization on Attributed Graphs", accepted by the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). DGBO/PDGBO methods can deal with attributed graphs. It prevents the cubical complexity of the GPs by adopting a deep graph neural network to surrogate black-box functions, and can scale linearly with the number of observations. Applications include molecular discovery and urban road network design.

Two deep surrogate models are contained in this code, i.e., "DGBO_GC-batch" is the DGBO with Graph Convolution (DGBO_{GC}), and "DGBO_GN-batch" is the DGBO with Graph Networks (DGBO_{GN}).

If you want to run this code, you should ensure that you have installed the following packages:

tensorFlow
spicy
pickle
numpy
emcee
networkx

#for graphNet
graph_nets
sonnet

#optional packages:
sklearn
bayes_opt ##via "pip install bayesian-optimization" to install. The website of this package is https://github.com/fmfn/BayesianOptimization

(Please see the "dependency_packages.md" in their respective folders to find the detailed versions of these packages.)

After you installed all dependency packages, choose and go into "DGBO_GC-batch" or "DGBO_GN-batch" folder, and then you can run DGBO with the default setting as:

$$ python DGBO.py --run=True

, or you can see the help message by running as:

$$ python DGBO.py -h

Note: If you try the zinc dataset, you should run “genConvMolFeatures.py” in “rdkit_preprocessing/” to convert SMILES strings to attributed graphs including xxx-attr.pkl, xxx-graph.pkl, and xxx-label.pkl.

scalable-and-parallel-dgbo's Issues

运行DGBO_GC-batch

崔老师您好，我在学习您的DGBO_GC方法，我采用的overfit_type是默认的opt。目前程序运行初始化参数，代码如下：
def initialization(data,n,info):
print ("-----------initialization-----------")
xlist=np.linspace(0,len(data.candidates)-1,len(data.candidates))
xlist = xlist.tolist()
xlistinit=[]
if len(data.candidates)%n==0:
delta=len(data.candidates)/n
else:
delta=len(data.candidates)/n+1
for i in range(n):
xlistinit.append(random.sample(xlist[i*int(delta):min((i+1)*int(delta),len(data.candidates))],1)[0])
xlist=xlistinit
xlist=np.array(xlist,dtype=np.integer)
for i in range(n):
print ("initialization [%s] ..."%(i))
y=evaluate(info,data.candidates[xlist[i]])
info.add(xlist[i],y)
if info.overfit_type=="opt":
print ("init the setting of params on overfitting...")
bo = BayesianOptimization(initsettingCV,{'dropout_': (0,1), 'weight_decay_': (1e-5,1e-1)},random_state=rng)
bo.maximize(init_points=21, n_iter=30, acq='ei')
print (bo.res['max'])
在上述代码中，运行到 bo = BayesianOptimization(initsettingCV,{'dropout_': (0,1), 'weight_decay_': (1e-5,1e-1)},random_state=rng)后进入了5折交叉验证，但是一直循环交叉验证的过程，结果如下：
| 2 | -1.044 | 0.9985 | 0.01969 |
[cv 1] [ 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] [0 1 2 3]
add a model
[cv 2] [ 0 1 2 3 8 9 10 11 12 13 14 15 16 17 18 19] [4 5 6 7]
add a model
[cv 3] [ 0 1 2 3 4 5 6 7 12 13 14 15 16 17 18 19] [ 8 9 10 11]
add a model
[cv 4] [ 0 1 2 3 4 5 6 7 8 9 10 11 16 17 18 19] [12 13 14 15]
add a model
[cv 5] [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15] [16 17 18 19]
add a model
transition_matrix: (16, 257)
train: features_comb shape is : (257, 68)
train: constructing adj...
Build DGCN_Comb2...
Architecture: { gcn[1 2 3 4 5]->concat(0.8)->dnn[1 2 3 4 5] }
heur: dropout=0.16990029785933825, weight_decay=0.047577751206738225
Start to train network.
Epoch: 0001 train_loss= 45.74941 time= 1.25621s total_time= 1.25621s
Epoch: 0101 train_loss= 33.50009 time= 0.00600s total_time= 1.88035s
Train net Finished!
acc= 1.3567712
transition_matrix: (16, 257)
train: features_comb shape is : (257, 68)
train: constructing adj...
Build DGCN_Comb2...
Architecture: { gcn[1 2 3 4 5]->concat(0.8)->dnn[1 2 3 4 5] }
heur: dropout=0.16990029785933825, weight_decay=0.047577751206738225
Start to train network.
Epoch: 0001 train_loss= 53.56653 time= 1.30929s total_time= 1.30929s
Epoch: 0101 train_loss= 33.76947 time= 0.00700s total_time= 1.95644s
Train net Finished!
acc= 1.1174575
transition_matrix: (16, 257)
train: features_comb shape is : (257, 68)
train: constructing adj...
Build DGCN_Comb2...
Architecture: { gcn[1 2 3 4 5]->concat(0.8)->dnn[1 2 3 4 5] }
heur: dropout=0.16990029785933825, weight_decay=0.047577751206738225
Start to train network.
Epoch: 0001 train_loss= 51.28548 time= 1.23025s total_time= 1.23025s
Epoch: 0101 train_loss= 33.82351 time= 0.00700s total_time= 1.86339s
Train net Finished!
acc= 0.5193358
transition_matrix: (16, 257)
train: features_comb shape is : (257, 68)
train: constructing adj...
Build DGCN_Comb2...
Architecture: { gcn[1 2 3 4 5]->concat(0.8)->dnn[1 2 3 4 5] }
heur: dropout=0.16990029785933825, weight_decay=0.047577751206738225
Start to train network.
Epoch: 0001 train_loss= 45.57104 time= 1.31298s total_time= 1.31298s
Epoch: 0101 train_loss= 32.98197 time= 0.00700s total_time= 1.97312s
Train net Finished!
acc= 1.7546887
transition_matrix: (16, 257)
train: features_comb shape is : (257, 68)
train: constructing adj...
Build DGCN_Comb2...
Architecture: { gcn[1 2 3 4 5]->concat(0.8)->dnn[1 2 3 4 5] }
heur: dropout=0.16990029785933825, weight_decay=0.047577751206738225
Start to train network.
Epoch: 0001 train_loss= 47.50560 time= 1.25738s total_time= 1.25738s
Epoch: 0101 train_loss= 33.11481 time= 0.00600s total_time= 1.89652s
Train net Finished!
acc= 1.9915786
上述过程一直循环，无法运行到 bo.maximize(init_points=21, n_iter=30, acq='ei')这一步。不知道问题出现在哪里。
另外发现当不debug，直接运行整个项目，当迭代次数为51之后会跳到bo.maximize(init_points=21, n_iter=30, acq='ei')这一步，但是会报错。错误如下：Traceback (most recent call last):
File "E:/model/Scalable-and-Parallel-DGBO-master/Scalable-and-Parallel-DGBO-master/DGBO_GC-batch/DGBO.py", line 338, in
BO_process(data,params,info)
File "E:/model/Scalable-and-Parallel-DGBO-master/Scalable-and-Parallel-DGBO-master/DGBO_GC-batch/DGBO.py", line 210, in BO_process
initialization(data,params.initn,info)
File "E:/model/Scalable-and-Parallel-DGBO-master/Scalable-and-Parallel-DGBO-master/DGBO_GC-batch/DGBO.py", line 135, in initialization
print (bo.res['max'])
TypeError: list indices must be integers or slices, not str。
这个问题困扰我很久了，一直不知道原因，希望您百忙之中可以帮忙给解答一下。我将感激不尽，谢谢您~