ebenmichael / side-projects Goto Github PK

I'm recently working on programming a DQN, your code offers me great help. However, I think your DQN doesn't work. For example, when I ran the tit for tat prison's dilemma, I found ipd_feats(state,action) is not applicable for self.feat_funct(state) in class DQN. That's trivial, I try to modify the code. Here comes problem about Keras which I could hardly figure out:
could you help me with this, what's your keras version?

/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py:484: UserWarning: theano.function was asked to create a function computing outputs given certain inputs, but the provided input variable at index 1 is not part of the computational graph needed to compute the outputs: keras_learning_phase. To make this warning into an error, you can pass the parameter on_unused_input='raise' to theano.function. To disable it completely, use on_unused_input='ignore'. **kwargs) Traceback (most recent call last): File "qlearning_FA_NN.py", line 644, in <module> ipdTft(100,0.9,0.1,alpha = .8) File "qlearning_FA_NN.py", line 494, in ipdTft q1.qUpdate(prevState,qa1,reward[0],newState) File "qlearning_FA_NN.py", line 345, in qUpdate y[i] = reward + self.gamma * self.maxOldQ(next_state) File "qlearning_FA_NN.py", line 384, in maxOldQ q = self.oldQ(feat,a) File "qlearning_FA_NN.py", line 320, in oldQ return(self.old_learn.predict(f)) File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 444, in predict return self.model.predict(x, batch_size=batch_size, verbose=verbose) File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1081, in predict self._make_predict_function() File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 666, in _make_predict_function **self._function_kwargs) File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 498, in function return Function(inputs, outputs, updates=updates, **kwargs) File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 484, in __init__ **kwargs) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function.py", line 320, in function output_keys=output_keys) File "/usr/local/lib/python2.7/dist-packages/theano/compile/pfunc.py", line 479, in pfunc output_keys=output_keys) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1776, in orig_function output_keys=output_keys).create( File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1428, in __init__ accept_inplace) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 177, in std_fgraph update_mapping=update_mapping) File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 171, in __init__ self.__import_r__(output, reason="init") File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 360, in __import_r__ self.__import__(variable.owner, reason=reason) File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 474, in __import__ r) theano.gof.fg.MissingInputError: ("An input of the graph, used to compute DimShuffle{x,x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.", keras_learning_phase)