muhanzhang / d-vae Goto Github PK

Hey, I got the following error when i try to run bayesian optimization in ENAS. Any ideas how to fix this?

2020-08-27 09:45:05.767860: E tensorflow/core/common_runtime/executor.cc:623] Executor failed to create kernel. Invalid argument: Default AvgPoolingOp only supports NHWC on device type CPU
[[{{node child_1/layer_1/pool_at_1/from_0/AvgPool}} = AvgPoolT=DT_FLOAT, data_format="NCHW", ksize=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 2, 2], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Traceback (most recent call last):
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Default AvgPoolingOp only supports NHWC on device type CPU
[[{{node child_1/layer_1/pool_at_1/from_0/AvgPool}} = AvgPoolT=DT_FLOAT, data_format="NCHW", ksize=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 2, 2], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "bo.py", line 268, in
score = -eva.eval(arc)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/evaluation.py", line 260, in eval
return self.ops["eval_func"](self.sess, "valid", feed_dict={self.ops["controller"]["sample_arc3"]: np.asarray(arch_str)})
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py", line 715, in customized_eval_once
acc = self.eval_once(sess, eval_set, feed_dict, verbose)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/models.py", line 179, in eval_once
acc = sess.run(acc_op, feed_dict=feed_dict)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 671, in run
run_metadata=run_metadata)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1255, in run
raise six.reraise(*original_exc_info)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1240, in run
return self._sess.run(*args, **kwargs)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1312, in run
run_metadata=run_metadata)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1076, in run
return self._sess.run(*args, **kwargs)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Default AvgPoolingOp only supports NHWC on device type CPU
[[node child_1/layer_1/pool_at_1/from_0/AvgPool (defined at /home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py:145) = AvgPoolT=DT_FLOAT, data_format="NCHW", ksize=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 2, 2], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'child_1/layer_1/pool_at_1/from_0/AvgPool', defined at:
File "bo.py", line 130, in
eva = Eval_NN() # build the network acc evaluater
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/evaluation.py", line 304, in Eval_NN
eva = Eval()
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/evaluation.py", line 234, in init
self.ops = self.get_ops(images, labels)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/evaluation.py", line 178, in get_ops
child_model.connect_controller(controller_model)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py", line 736, in connect_controller
self._build_valid()
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py", line 648, in _build_valid
logits = self._model(self.x_valid, False, reuse=True)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py", line 222, in _model
layer, out_filters, 2, is_training)
File "/home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py", line 145, in _factorized_reduction
x, [1, 1, 1, 1], stride_spec, "VALID", data_format=self.data_format)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 2110, in avg_pool
name=name)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 72, in avg_pool
data_format=data_format, name=name)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/anaconda/envs/azureml_py36_automl/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Default AvgPoolingOp only supports NHWC on device type CPU
[[node child_1/layer_1/pool_at_1/from_0/AvgPool (defined at /home/mchatzi/my_projects/D-VAE/bayesian_optimization/../software/enas/src/cifar10/eval_child.py:145) = AvgPoolT=DT_FLOAT, data_format="NCHW", ksize=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 2, 2], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Hey,
Is there a chance to upload the pretrained ENAS model, for the 12 layers?

Thanks in advance.

Hi
I just run the line

python train.py --data-type ENAS --reprocess --data-name final_structures6 --save-interval 10 --save-appendix _SVAE --epochs 300 --lr 1e-4 --model SVAE --bidirectional --nz 56 --batch-size 32

and get approximately 0 test accuracy, the same is for --model DVAE. Could you rerun it and confirm that your code works ok, please?

I have run the expirements with DVAE and in bayesian optimization i get the following results in best_arc_scores.txt :

Is this metric accuracy? Why is it nearly 10% ?
Thank you in advance.

Could you please give more details about how to reproduce Table 2 in the paper?
The repository contains train commands for the models and scripts for the Bayesian optimization. But for the Table 2 experiments, it seems that SGP has to be trained differently? There is the following option in the bo.py code, but I do not find the sh file. Thank you already!

if args.predictor:
copy('run_pred_{}.sh'.format(data_type), save_dir)

Hi,

Thanks for your great work! When I read your paper and code, I am curious about the (bidirectional) encoding part. In your paper, the aggregation for the source nodes is described as:

If an empty set is input to A (corresponding to the case for the starting node without any predecessors), we let A output an all-zero vector.

This case applied to both the forward layer and the backward layer. My point is that after the forwarding, the hidden states of the last nodes are obtained, can it be the initial values of A for the backward layer?

I know this way is not consistent with the formulation Eq. 3, wherein the incoming information is defined as aggregated information from the predecessors. But if the computation is done in this way, actually we can introduce the iterative scheme into the architecture:

0s for source node -> (forward layer) -> Some vectors for sink node -> (backward layer) -> Some vectors for source node -> (forward layer) -> ......

The benefit could be that after a few iterations, the hidden state of source nodes and sink nodes become converged. And it may get a better encoding space for the later decoding. I think such an iterative scheme is used in the following two papers (all related to Satisfiability problem):

1. Amizadeh, Saeed, Sergiy Matusevych, and Markus Weimer. "Learning to solve circuit-SAT: An unsupervised differentiable approach." International Conference on Learning Representations. 2018.
2. Learning a SAT Solver from Single-Bit Supervision.

Looking forward to your feedback. Thanks!

Since the graphs in the two datasets have very similar sizes, I wonder why you use these very different batch sizes for the two tasks?